Device and method for analyzing state of manual work by worker, and work analysis program

ABSTRACT

A device for analyzing a worker&#39;s work state, the analysis involving generation of determination data for determining whether worker&#39;s manual work is performed in a predetermined work order. This device includes imaging and setting units. The imaging unit images, a work video, a worker&#39;s manual working state in which the worker repeatedly performs predetermined work. This predetermined work is performed by repeating a plurality of fundamental work operations by hand. Based on a predetermined motion which is previously set for each of the fundamental work operations, the setting unit sets delimitation information to delimit the work video every fundamental work operation at a timing when the predetermined motion is detected. In the device, a generation unit generates determination data which includes both the imaged work video and the set delimitation information. Based on the determination data, the worker&#39;s manual work can be analyzed in various ways.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims the benefit of priority fromearlier Japanese Patent Applications No. 2019-024072 filed on Feb. 14,2019; No. 2019-035209 filed on Feb. 28, 2019; No. 2019-043619 filed onMar. 11, 2019; No. 2019-065318 filed on Mar. 29, 2019; and No.2019-147092 filed on Aug. 9, 2019 the descriptions of which areincorporated herein by reference.

BACKGROUND Technical Field

The present disclosure relates to devices and methods for analyzing thestate of manual work by a worker, for example, by generatingdetermination data for determining whether or not the manual work isperformed by a worker according to a predetermined procedure, andfurther relates to work analysis programs.

Related Art

There have been systems for supporting work by using determination data(annotation data) for determining whether or not correct work isperformed. The determination data are obtained by acquiring a largenumber of video images of the worker performing the predetermined worksuch as assembling a plurality of parts to an assembly target such as asubstrate, and analyzing the video images thus acquired.

For example, a work analysis device disclosed in JP 2017-134666 A isknown as a technique related to generation of such annotation data. Inthe work analysis processing performed by the work analysis device, thework video image that is read is played back and displayed for analysison the work analysis screen together with the measurement state displayareas and the like. When the worker performs the delimitation operationby clicking a portion of the playback screen with a mouse duringplayback of the video image, a selection screen is displayed on theportion of the playback screen. In this display state, when the workerperforms a selection operation by clicking any of the attributeinformation displayed on the selection screen with a mouse, the analysisdata including the attribute information selected by the selectionoperation is associated with the video image range delimited by thedelimitation operation, and recorded.

Further, at the manufacturing site, there may be a case where manualassembly work is performed on, for example, printed circuit boards ordevices in the course of manufacturing. In order to improve workefficiency of such work, a method is known for performing work analysisby imaging predetermined repetitive work to generate a work video imagefor work analysis, and analyzing the generated work video image. Forexample, a moving pattern specifying device disclosed in JP 2008-108008A is known as a technique related to such work analysis.

In the moving pattern specifying device, the moving trajectory of themarkers respectively provided on the right hand and the left hand of theworker are obtained from the video image data as standard time seriesdata. Among these standard time series data, points that can be regardedas candidates for delimiter of work are extracted as delimiter candidatepoints. When the delimiter position information indicating a delimiterposition and the delimiter content information corresponding to thedelimiter position are specified in response to an input instructioninputted from a user who has watched the video image data near thedelimiter candidate point, the type of work and the start time and endtime of the work can be specified on the basis of the information thusspecified.

Moreover, in an image extraction analysis device disclosed in JP2007-243846 A, images including specified image patterns are extractedfrom the video image obtained by imaging the work site, and the timeintervals at which the images repeatedly appear are calculated. Then,the time intervals are grouped on the basis of the time intervals withsubstantially the same appearance frequency, and are displayed on thescreen as a histogram. As the image and the time interval of the groupspecified by the user are displayed on the screen in accordance with theabove display, the delimiter of the step is displayed together with thevideo image and time interval. Accordingly, without sequentiallywatching the video images obtained by imaging the work site, it ispossible to easily grasp which work takes time in each step.

As described above, at the manufacturing site, there may be a case wheremanual assembly work is performed on, for example, printed circuitboards or devices in the course of manufacturing. Although such work isperformed according to a predetermined work procedure, some parts may bemissed to be assembled or wrong parts may be assembled since such workusually involves a plurality of steps, and, in the case of assemblywork, a plurality of types of parts are assembled in general. Therefore,it has been proposed, for example in JP 2010-211623 A, to provide adetection unit such as a weight sensor on the parts box in order todetermine whether or not a part taken out of the parts box is suitablefor the work procedure.

In this case, when the work is performed on different products, thetypes and amount of parts to be assembled are also different. As aconsequence, there are problems that providing the above detection uniton the parts box increases the facility cost in proportion to the amountof parts, and that a great deal of labor is required for installation ofthe detection unit and association with the work procedure. In ordersolve the problems, for example, a work support device disclosed in JP2018-156279 A is known. In this work support device, a monitoring area,which is a range for monitoring the work, is set in the imaging range ofthe imaging unit by input operation according to the work procedure.Then, a portion corresponding to the monitoring area of the acquiredimage is compared with a portion corresponding to the monitoring area inanother image that is acquired before the above image to perform dynamicdetection for the monitoring area. On the basis of the result of dynamicdetection, it is determined whether or not the work procedure has beenfollowed, and the determination result is notified to the worker.

In the work analysis device disclosed in JP 2017-134666 A, after a videoimage of the repetitive predetermined work, in which a plurality offundamental work operations (in this publication, simply referred to as“operations”) are repeatedly performed in a predetermined order, isacquired, an operation such as clicking a portion of the playback screenis required for the worker or the like watching the playback of thevideo image. JP 2017-134666 A describes that the “work” is a set of “aplurality of operations” which are performed as worker's manualoperations under a previously planned schedule or a specified object.

For this reason, the worker or the like cannot delimit the video imagerange for each fundamental work operation unless without watching thewhole video image, which is burdensome for the worker. In particular,when there are an increased number of types of fundamental workoperations and an increased number of repetition of the predeterminedwork, the above problem becomes more apparent. Further, since it isnecessary to perform an operation in parallel with watching the playbackof the video image after the video image of the repetition of thepredetermined work is obtained, there is a first problem that annotationdata cannot be generated in parallel with acquiring the video image ofthe predetermined work.

In addition, for example, when the work to be analyzed is to assemblethe parts accommodated in the parts box to an assembly target, the workvideo image can be delimited for each fundamental work operation bydetermining when a part is taken out of the parts box. However, forexample, if the parts to be taken out from the parts box are small, thedetermination whether the part has been taken out of the parts box ornot becomes ambiguous, leading to a reduced accuracy in delimitation ofthe work video image.

Furthermore, according to the work support device disclosed in JP2018-156279 A, in setting of the monitoring areas corresponding to therespective parts boxes, the worker is required to perform the work ofdrawing a frame surrounding the parts box in the image by using awireless mouse while checking the position of the parts box in the imageacquired by the imaging unit. This is because the monitoring areascannot be set in advance since the types and amount of the parts boxesas well as the positions of the parts boxes may vary depending on themanufacturing lot. As a consequence, since the worker must operate awireless mouse in parallel with watching the screen for eachmanufacturing lot, there is a second problem that an initial settingwork for setting the monitoring areas is time-consuming, and a wrongmonitoring area may be set due to an operation error of the worker orthe like.

SUMMARY

The present disclosure is basically directed to analyze the state ofwork manually performed by the worker, for example, part assembly work(manual work), in more efficient and more accurate manner. Particularly,in addition to the above basic object, a first object of the presentdisclosure is to generate determination data (work video image) fordetermining whether or not the work (manual work) manually performed bythe worker follows a predetermined procedure.

Further, in addition to the above basic object, a second object of thepresent disclosure is to set a monitoring area easily and correctlywithout imposing a work burden on the worker.

In order to achieve the first object, an exemplary embodiment of thefirst mode relates to a work analysis device which generatesdetermination data for determining whether or not a worker's manual workis now repeatedly performed according to a predetermined work procedure,the work analysis device comprising:

-   -   an imaging unit imaging, as a work video, a state where a        plurality of fundamental work operations are performed        repeatedly and manually in a predetermined order by the worker;    -   a setting unit setting delimitation information for delimiting        the work video at detection timings at each of which a        predetermined motion is detected, based on the predetermined        motion which is previously set for each of the fundamental work        operations; and    -   a generation unit generating the determination data which        includes both the work video and the delimitation information.

In the present disclosure, when it is assumed that target work (or task)is performed under a predetermined plan, the target work issegmentalized into a plurality of elemental work operations, which canbe called fundamental work operations. When assuming a simple targetwork (task) that parts are assembled to a printed circuit board,collecting parts from a parts box, arranging the collected parts at anassembly position on the board, or others correspond to fundamental workoperations. For this reason, the “fundamental work operation” can bedefined according to conditions or attributions of different work

Hence, in the foregoing basic configuration, worker's predeterminedmotions for the fundamental work operations are optically imaged. Onlythis imaging makes it possible to automatically delimit the work videoevery fundamental work operation. Hence, determination data which arefor determining whether or not the “correct” work (or operation)according to the previously set work order can be generated in real timebased on the work video. For this reason, the state of various types ofmanual work, such as assembly of parts to an assembly target by hand,can be analyzed more efficiently and accurately.

Besides the foregoing work analysis device, there is also provided amethod of analyzing the work involving a plurality of fundamental workoperations which are performed manually by the worker. The methodincludes steps functionally equivalent to the components of the workanalysis devices.

In order to achieve the first object, an exemplary embodiment of thesecond mode relates to a work analysis device which generates a workvideo for a work analysis by repeatedly imaging a predetermined work inwhich a plurality of fundamental work operations are repeatedlyperformed with worker's manual work in a predetermined order, the devicecomprising:

-   -   an imaging unit imaging the predetermined work repeatedly        performed by the worker;    -   a monitoring area setting unit setting a plurality of monitoring        areas including a first monitoring area for detecting a first        operation among the fundamental work operations and a second        operation among the fundamental work operations, for each of the        fundamental work operations in an imaging range of the imaging        unit;    -   a reliability setting unit setting, for each of the monitoring        areas, reliability such that the reliability increases with an        increase in a possibility that a motion related to the        fundamental work operations is performed in the monitoring area,        based on a comparison made between a portion corresponding to        the monitoring area in an image captured by the imaging unit and        a portion corresponding to the monitoring area in a further        image captured by the imaging unit, the further image being        captured prior to the image;    -   a determination unit determining whether or not a mutually        corresponding fundamental work operation among the fundamental        work operations is performed, based on the reliabilities which        are set to the first and second monitoring areas; and    -   a delimitation information setting unit setting delimitation        information which enables a work video imaged by the imaging        unit to be delimited every fundamental work operation determined        to be perfumed by the determination unit, based on at least one        of a timing at which the image for setting the reliability is        set for the first monitoring area and a timing at which the        image for setting the reliability is set for the second        monitoring area.

According to this configuration, for each of the first and secondmonitoring areas, reliability is set based on a comparison made betweena portion corresponding to the monitoring area in an image captured bythe imaging unit and a portion corresponding to the monitoring area in afurther image captured by the imaging unit, the further image beingcaptured prior to the image. Hence, the accuracy of determining thefundamental work operations can be raised, thereby realizingdelimitation of the work video in a higher accuracy. This will lead to amore efficient and accurate analysis of the manual work.

In order to achieve the second object, an exemplary embodiment of thethird mode relates to a work support device which supports workperformed in a predetermined work procedure, the work involving takingout parts accommodated in a plurality of parts boxes, the devicecomprising:

-   -   an imaging unit;    -   a monitoring area setting unit setting monitoring areas        respectively to the parts boxes within an imaging range of the        imaging unit;    -   a detection section detecting a worker's taking action of the        parts from each of the parts boxes, based on a comparison made        between a portion corresponding to the monitoring area in a        designated image captured by the imaging unit and a portion        corresponding to the monitoring area in a further image captured        by the imaging unit, the further image being captured prior to        the designated image;    -   a determination unit determining whether or not the work        according to the predetermined work procedure is performed,        based on a detection result provided by the detection section;        and    -   a notification unit notifying the worker of a determined result        provided by the determination unit,    -   wherein the monitoring area setting unit is configured to set,        based on an image captured by the imaging unit, the monitoring        areas every one of the parts boxes, the monitoring aeras        corresponding to the parts boxes which are moved in a        predetermined movement state, when the parts boxes are        individually moved in the predetermined movement state.

In this configuration, the worker moves the parts boxes in apredetermined movement state before the work for example, and themonitoring area for each parts box can be set easily. There is no needto operate the mouse on a display like a conventional manner, thuslighting worker's work burden. Hence, the monitoring areas can be setmore easily and accurately, thus improving a work analysis in terms ofefficiency and accuracy.

In order to achieve the second object, an exemplary embodiment of thefourth mode relates to a work support device which supports workperformed in a predetermined work procedure, the work involving takingout parts accommodated in a plurality of parts boxes, the devicecomprising:

-   -   an imaging unit;    -   a monitoring area setting unit setting monitoring areas        respectively to the parts boxes within an imaging range of the        imaging unit;    -   a detection section detecting a worker's taking action of the        parts from each of the parts boxes, based on a comparison made        between a portion corresponding to the monitoring area in a        designated image captured by the imaging unit and a portion        corresponding to the monitoring area in a further image captured        by the imaging unit, the further image being captured prior to        the designated image;    -   a determination unit determining whether or not the work        according to the predetermined work procedure is performed,        based on a detection result provided by the detection section;        and    -   a notification unit notifying the worker of a determined result        provided by the determination unit, wherein    -   each of the parts boxes has a peripheral wall having an upper        end face, the upper end face being formed into a polygonal ring        shape with a plurality of corners, and    -   the monitoring area setting unit is configured to i) detect a        boundary in an image captured by the imaging unit, the boundary        starting from a stat point designed by a worker's finger touched        to one of the corners, extending along line segments of the        polygonal ring shape with a turn at each of a plurality of        intersections configured by two of the corners, and return to        the start point, and ii) set a polygonal ring shaped area        surrounded by the boundary as the monitoring area for each of        the parts boxes.

Hence, before the manual work, the worker can touch his or her finger toone corner on the upper end face of the peripheral wall of each partsbox under imaging of the imaging unit. This simple action enablessetting of the monitoring area for each parts box. Accordingly,similarly to the foregoing advantages, there is no need to operate themouse on a display like a conventional manner, thus lighting worker'swork burden. Hence, the monitoring areas can be set more easily andaccurately, thus improving a work analysis in terms of efficiency andaccuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a view illustrating a schematic configuration of a workanalysis device according to a first embodiment.

FIG. 2 is a view illustrating an imaging state in which respective partsboxes are imaged.

FIG. 3 is a block diagram illustrating an electric configuration of awork analysis device.

FIG. 4A is a view illustrating a delimitation motion of a fundamentalwork operation A in the first embodiment.

FIG. 4B is a view illustrating a delimitation motion of a fundamentalwork operation B in the first embodiment.

FIG. 5A is a view illustrating a delimitation motion of a fundamentalwork operation C in the first embodiment.

FIG. 5B is a view illustrating a delimitation motion of a fundamentalwork operation D in the first embodiment.

FIG. 6 is a flowchart illustrating the order of execution in workanalysis processing performed by a control unit.

FIG. 7 is a view illustrating work delimitation detection results inwhich delimitation information is set for each fundamental workoperation.

FIG. 8 is a view illustrating a monitoring area in which a delimitationmotion is detected according to a second embodiment.

FIG. 9A is a view illustrating a delimitation motion of a fundamentalwork operation A in the second embodiment.

FIG. 9B is a view illustrating a delimitation motion of a fundamentalwork operation B in the second embodiment.

FIG. 10A is a view illustrating a delimitation motion of a fundamentalwork operation C in the second embodiment.

FIG. 10B is a view illustrating a delimitation motion of a fundamentalwork operation D in the second embodiment.

FIG. 11 is a view illustrating a monitoring device of a work analysisdevice according to a third embodiment.

FIG. 12 is a view illustrating that a delimitation motion is detected inthe third embodiment.

FIG. 13A is a view illustrating a monitoring device of a work analysisdevice according to a fourth embodiment.

FIG. 13B is a view illustrating that a parts box is positioned in anobservation area of FIG. 13A.

FIG. 14A is a view illustrating a monitoring device of a work analysisdevice according to a modified example of the fourth embodiment.

FIG. 14B is a view illustrating that a parts box is positioned in anobservation area of FIG. 14A.

FIG. 15 is a view illustrating that a monitoring area is set in a fifthembodiment.

FIG. 16 is a view illustrating a height state in which detection of adelimitation motion is valid and a height state in which detection of adelimitation motion is invalid in the fifth embodiment.

FIG. 17A is a view illustrating a state in which a part 20 a is takenout of a part box 30 a.

FIG. 17B is a view illustrating a state in which a part 20 a isassembled to a workpiece W.

FIG. 18 is a graph showing a change in weight monitored by a monitoringdevice that monitors the states of FIGS. 17A and 17B.

FIG. 19 is a view illustrating setting of a temporary monitoring area ina parts box in a seventh embodiment.

FIG. 20 is a view illustrating setting of a temporary monitoring area ona workpiece in the seventh embodiment.

FIG. 21A is a view illustrating a monitoring area set by a temporarymonitoring area weighted with “3.”

FIG. 21B is a view illustrating a monitoring area set by a temporarymonitoring area weighted with “2” or more.

FIG. 22 is a view illustrating a schematic configuration of a workanalysis device according to an eighth embodiment.

FIG. 23 is a flowchart illustrating the order of execution in monitoringarea setting performed by a control unit in the eighth embodiment.

FIG. 24 is a flowchart illustrating the order of execution in asubroutine of parts box relative coordinate estimation in FIG. 23.

FIG. 25 is a flowchart illustrating the order of execution in asubroutine of camera relative coordinate estimation in FIG. 23.

FIG. 26A is a view illustrating a positional relationship between asecond imaging unit and a parts box.

FIG. 26B is a view illustrating an image captured by the second imagingunit in the positional relationship of FIG. 26A.

FIG. 27 is a view illustrating a method of calculating a distance from asecond imaging unit to a parts box.

FIG. 28A is a view illustrating an attachment position of a parts boxcode to a parts box.

FIG. 28B is a view illustrating an attachment position of a parts boxcode to a parts box.

FIG. 28C is a view illustrating an attachment position of a parts boxcode to a parts box.

FIG. 29A is a view illustrating that a parts box code is randomlyprovided on a parts box.

FIG. 29B is a view illustrating an image captured by the second imagingunit in the positional relationship of FIG. 29A.

FIG. 30 is a view illustrating an essential part of a work analysisdevice according to a first modified example of the eighth embodiment.

FIG. 31 is a view illustrating an essential part of a work analysisdevice according to a second modified example of the eighth embodiment.

FIG. 32A is a view illustrating that a parts box code is provided on anupper end face of a peripheral wall of a parts box.

FIG. 32B is a view illustrating that a parts box code is provided on anupper lid.

FIG. 33 is a view illustrating an essential part of a work analysisdevice according to a ninth embodiment.

FIG. 34A is a view illustrating a distance image in a state in which aparts box is imaged near an imaging unit.

FIG. 34B is a view illustrating a distance image in a state in which aparts box is imaged at a position farther away from a position in FIG.34A.

FIG. 35 is a flowchart illustrating the order of execution in monitoringarea setting performed by a control unit in the ninth embodiment.

FIG. 36A is a view illustrating a captured image of two parts boxesplaced on a shelf.

FIG. 36B is a view illustrating that FIG. 36A is divided into blocks.

FIG. 37 is a graph of frequency features extracted from respectiveblocks B1 to B4 of FIG. 36B.

FIG. 38A and FIG. 38B is a view illustrating that two parts boxes areplaced on a shelf.

FIG. 39A is a view illustrating an essential part of a work analysisdevice according to a twelfth embodiment, in which this view illustratesa captured image of two parts boxes placed on a shelf.

FIG. 39B is a view illustrating an essential part of a work analysisdevice according to the twelfth embodiment, in which this viewillustrates a captured image in which an area of a second color isextracted.

FIG. 39C is a view illustrating an essential part of a work analysisdevice according to the twelfth embodiment, in which this viewillustrates a captured image after filtering is performed.

FIG. 40 is a view illustrating an essential part of monitoring areasetting performed by a work analysis device according to a thirteenthembodiment.

FIG. 41 is a flowchart illustrating the order of execution in monitoringarea setting performed by a control unit in the thirteenth embodiment.

FIG. 42 is a flowchart illustrating the order of execution in asubroutine of parts area setting in FIG. 41.

FIG. 43 is a view illustrating a monitoring area set by a parts area.

FIG. 44 is a view illustrating an essential part of monitoring areasetting performed by a work analysis device according to a firstmodified example of the thirteenth embodiment.

FIG. 45 is a view illustrating a detection result obtained byline-scanning a luminance in the X coordinate direction with respect toa predetermined Y coordinate.

FIG. 46 is a view illustrating an essential part of monitoring areasetting performed by a work analysis device according to a secondmodified example of the thirteenth embodiment.

FIG. 47 is a graph showing the frequency of appearance of the angle ofcorner obtained by corner detection in an extraction image.

FIG. 48 is a view illustrating that monitoring areas are set by usingmarkers in an image in which parts boxes are imaged.

FIG. 49A is a view illustrating a state before parts are assembled to aworkpiece, together with the respective monitoring areas.

FIG. 49B is a view illustrating a state after parts are assembled to aworkpiece, together with the respective monitoring areas.

FIG. 50 is a flowchart illustrating the order of execution in workanalysis processing performed by a control unit.

FIG. 51 is a flowchart illustrating the order of execution in firstreliability setting in FIG. 50.

FIG. 52 is a flowchart illustrating the order of execution in secondreliability setting in FIG. 50.

FIG. 53 is a diagram illustrating a total reliability set on the basisof the first reliability and the second reliability.

FIG. 54 is a view illustrating a schematic configuration of a worksupport device according to a fifteenth embodiment.

FIG. 55 is a block diagram illustrating an electric configuration of awork support device.

FIG. 56 is a flowchart illustrating the order of execution in worksupport performed by a control unit in the fifteenth embodiment.

FIG. 57 is a flowchart illustrating the order of execution in monitoringarea setting in FIG. 56.

FIG. 58 is a view illustrating a swinging state of a parts box.

FIG. 59A is a view illustrating an imaging state of an image in which aparts box 30 a is determined as being stopped as it is returned to anoriginal position.

FIG. 59B is a view illustrating an imaging state of another imageacquired immediately before the image of FIG. 59A.

FIG. 60A is a view illustrating that a first monitoring area is set inan image.

FIG. 60B is a view illustrating that all the monitoring areas are set inan image.

FIG. 61 is a flowchart illustrating the order of execution in worksupport performed by a control unit in a sixteenth embodiment.

FIG. 62 is a view illustrating an imaging state in which respectiveparts boxes are imaged.

FIG. 63 is a flowchart illustrating the order of execution in worksupport performed by a control unit in a seventeenth embodiment.

FIG. 64 is a flowchart illustrating the order of execution in monitoringarea setting in FIG. 63.

FIG. 65 is a view illustrating that a search target is switched with acorner where a finger in a stationary state touches being taken as astart point.

FIG. 66 is a view illustrating a case where the direction of a turn atthe intersection in search is set to a first direction to therebyprevent influence of detecting unintended intersection.

FIG. 67 is a view illustrating a monitoring area that is set when twocorners thereof are touched by a finger in a stationary state in aneighteenth embodiment.

FIG. 68 is a view illustrating a monitoring area that is set when aninner edge of an upper end face of a parts box is set as a search targetin a nineteenth embodiment.

FIG. 69 is a flowchart illustrating the order of execution in monitoringarea setting in a twentieth embodiment.

FIG. 70 is a view illustrating a ring-shaped trajectory drawn by afinger of a worker tracing an upper end face of a parts box in atwentieth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

With reference to the accompanying drawings, various embodiments willnow be described.

First Embodiment

The following description will be given of devices and methods (workanalysis devices and work analysis methods) for analyzing information onwhether or not the manual work by the worker is performed according to apredetermined procedure, that is, whether or not the manual work is“correctly” performed, as well as obtaining information indicative ofthe state of the manual work. The description will be further given ofvarious modes of work analysis programs for computers for implementingthese devices and methods.

In the embodiments described herein, configurations and operations ofthe work analysis device will be described, and, in connection with thisdescription, work analysis methods and work analysis programs will alsobe described.

With reference to the drawings, a first embodiment in which a workanalysis device is implemented will now be described.

As shown in FIG. 1, a work analysis device 10 according to the presentembodiment is provided on a work table 1 or the like. The work analysisdevice 10 is configured to acquire a work video image of predeterminedmanual work, in which a plurality of fundamental work operations areperformed in a predetermined order, repeatedly performed by a worker M,and generate determination data (annotation data) for determiningwhether or not the manual work is performed according to thepredetermined order (i.e., correct procedure) for each fundamental workoperation on the basis of the work video image. That is, thedetermination data is generated by setting delimitation information fordelimiting the work video image for each fundamental work operation, inwhich the predetermined manual work is repeated. Thus, a plurality ofvideo image ranges in which respective fundamental work operations areimaged can be extracted from a determination data.

On the work table 1, manual assembly work is performed by the worker. Anassembly target (hereinafter, also simply referred to as a workpiece W)such as a printed circuit board is transported and placed on the worktable 1. The work table 1 has a shelf 2 on which a plurality of partsbox 30 are horizontally arranged side by side as viewed from the workerM.

The respective parts box 30 contain different types of parts 20 to beassembled to the workpiece W. As shown in FIG. 2, in the presentembodiment, four parts boxes 30 a to 30 d are arranged side by side onthe shelf 2 such that the parts box 30 a contains parts 20 a, the partsbox 30 b contains parts 20 b, the parts box 30 c contains parts 20 c,and the parts box 30 d contains parts 20 d. FIG. 2 is an enlarged viewof a portion near the parts boxes 30 a to 30 d in an image captured byan imaging unit 13, which will be described later. It should be notedthat the term “parts box” is merely an example name, and a variety ofnames may be used in different factories. In this example, various namessuch as parts case, parts container, parts basket, and parts bag arecollectively referred to as a parts box.

As shown in FIGS. 1 and 3, the work analysis device 10 includes acontrol unit 11, a storage unit 12, an imaging unit 13, a displayingunit 14, a light emitting unit 15, a speaker 16, an operation unit 17, acommunication unit 18, and the like. Among these, the control unit 11,the storage unit 12, and the communication unit 18 constitute aprocessor 10A.

The control unit 11 is mainly composed of a computer having a CPU(central processing unit) 11A, which is mainly responsible forarithmetic operations, and a main memory 11B as a work area, andperforms overall control of the work analysis device 10 and variouscalculations as well as work analysis processing as described later. Thestorage unit 12 includes known storage media such as an HDD (not shown)and a non-volatile memory, as necessary, in addition to a ROM (read onlymemory (e.g., EEPROM)) 12A and a RAM (random access memory) 12B.Further, application programs (hereinafter, also referred to as workanalysis programs), a predetermined database, and the like forperforming work analysis processing are pre-stored and available for thecontrol unit 11 (that is, CPU 11A).

In the present embodiment, the ROM 12A functions as a non-transitorycomputer-readable recording medium, and stores procedures for the aboveapplication programs and other control and processing programs as sourcecodes. The non-transitory computer-readable recording medium may also bea RAM of a type in which stored information is not lost.

The program is read out by the CPU 11A into a preset work area 11B forexecution. The RAM 12B is configured to temporarily store the data whichare being processed by the CPU 11A. Further, the main memory 11B iscomposed of a RAM.

It should be noted that the configuration of the processor 10A is merelyan example, and any configuration may also be used as long as it canexecute programs for required work analysis, control, and processing.For example, a configuration which includes a plurality of CPUs forperforming distributed control or establishing a redundant system mayalso be used. The CPU 11A is an element that is mainly responsible forperforming arithmetic calculations in the computer system, and may alsohave a different name as long as it has a similar function (for example,an arithmetic unit).

The imaging unit 13 is configured as a camera having a light receivingsensor (for example, C-MOS area sensor or CCD area sensor). In thepresent embodiment, the imaging unit 13 is separately provided from adevice main body 10 a which includes the control unit 11 (CPU 11A), thedisplaying unit 14, and the like, and is disposed in an upper part ofthe work table 1 so as to acquire a video image of the respective statesof the parts boxes 30 a to 30 d and the workpiece W in addition to thework state by the worker M. In the present embodiment, the imaging unit13 is configured to acquire a video image (sequence of still images),for example, at 30 frame per second, and store the acquired video imagein the storage unit 12 so as to be analyzed by the control unit 11.

The displaying unit 14 is a liquid crystal display, for example, and iscontrolled by the control unit 11 (CPU 11A) to display an image acquiredby the imaging unit 13, predetermined information, and the like. Thedevice main body 10 a is mounted on a rear plate or the like of the worktable 1 so that the worker M can see the display screen of thedisplaying unit 14.

The light emitting unit 15 includes an LED, for example, and iscontrolled by the control unit 11 (CPU 11A) and to control the color ofemitted light and lighting and flashing states. The light emitting unit15 is disposed at a position that can be seen by the worker M. Thespeaker 16 is formed of a known speaker or the like, and is controlledby the control unit 11 to emit a predetermined sound and variousnotification sounds such as an alarm sound.

The operation unit 17 is configured to output an operation signalcorresponding to a manual input operation from an operator to thecontrol unit 11. Upon receiving the operation signal, the control unit11 (CPU 11A) performs processing corresponding to the inputtedoperation. The communication unit 18 is configured as a communicationinterface that performs data communication with an external device suchas a higher level device, and configured to cooperate with the controlunit 11 to perform communication.

The following description will be given of the work analysis processingperformed on the basis of the work analysis program, which is performedby the control unit 11 (CPU 11A), when the worker M performspredetermined manual work by which parts contained in a plurality ofparts boxes are individually assembled to the workpiece W according to awork procedure of a predetermined manual work.

In the present embodiment, the predetermined manual work (hereinafter,also simply referred to as a “work (target work or target task)”) to beanalyzed is the work including a fundamental work operation A ofassembling the part 20 a in the parts box 30 a to the workpiece W, afundamental work operation B of assembling the part 20 b in the partsbox 30 b to the workpiece W, a fundamental work operation C ofassembling the part 20 c in the parts box 30 c to the workpiece W, and afundamental work operation D of assembling the part 20 d in the partsbox 30 d to the workpiece W in this order.

In the work analysis processing, on the basis of a predetermined motion(hereinafter, also referred to as a delimitation motion) which is presetfor each fundamental work operation, determination data is generated toset delimitation information for delimiting the work video imageacquired by the imaging unit 13 by each fundamental work operation at atiming when the delimitation motion is detected. In the presentembodiment, a range corresponding to the parts box 30 a in the imagingrange by the imaging unit 13 is preset as a monitoring area P1 a. Asshown in FIG. 4A, a motion that the hand of the worker M enters themonitoring area Pla is set as the delimitation motion for thefundamental work operation A. Similarly, ranges corresponding to theparts boxes 30 b to 30 d in the imaging range by the imaging unit 13 arepreset as monitoring areas P1 b to P1 d, respectively. As shown in FIG.4B, a motion that the hand of the worker M enters the monitoring area P1b is set as the delimitation motion for the fundamental work operationB. As shown in FIG. 5A, a motion that the hand of the worker M entersthe monitoring area P1 c is set as the delimitation motion for thefundamental work operation C. As shown in FIG. 5B, a motion that thehand of the worker M enters the monitoring area P1 d is set as thedelimitation motion for the fundamental work operation D. FIGS. 4 and 5illustrate enlarged views of the respective ranges corresponding to theparts boxes 30 a to 30 d in the imaging range by the imaging unit 13.

As can be understood, in the present embodiment, the fundamental workoperation (or unit work operation) shows each of the elemental wokeoperations required to perform a target work (task). In the presentembodiment, worker's actions for the work, which can be delimited byevents at each of which a worker's hand enters the monitoring area setfor each of the parts boxes, is called the fundamental work operation.

The monitoring areas P1 a to P1 d can be set as standard ranges, forexample, by placing the parts boxes 30 a to 30 d in position, or may beset on the basis of the image difference generated by continuouslycapturing the parts boxes 30 a to 30 d while they are individuallyswinging.

Hereinafter, referring to a flowchart of FIG. 6, the work analysisprocessing performed by the control unit 11 (CPU 11A) will be describedin detail.

The control unit 11 (CPU 11A) starts the work analysis processing when apredetermined start operation is performed to the operation unit 17. Inthe imaging at step S101 shown in FIG. 6, a work video image of theworker M is acquired by the imaging unit 13. When any of the abovedelimitation motions is detected while the work video image is beingacquired, it is determined as “Yes” in the determination process at stepS103. Then, in the delimitation information setting at step S105, thedelimitation information for delimiting the work video image at theabove detection timing is set. The delimitation information can includea fundamental work operation name specified by the delimitation motion,information on the detection time, and the like. When a predeterminedcompletion operation is not performed (No at S107), the steps from stepS103 onward are repeated. Further, the control unit 11 that performs theabove delimitation information setting can correspond to an example of a“setting unit.”

By repeating the steps from step S103 onward, the delimitationinformation is set for each fundamental work operation. Thus, thedetection results of work delimitation shown in FIG. 7 can be obtained.For example, in the kth cycle shown in FIG. 7, when a motion by whichthe hand of the worker M enters the monitoring area P1 a is detected ata time t1, the time t1 is set as a start timing of the fundamental workoperation A. Then, when a motion by which the hand of the worker Menters the monitoring area P1 b is detected at a time t2, the time t2 isset as a start timing of the fundamental work operation B and also as anend timing of the fundamental work operation A. Similarly, when a motionby which the hand of the worker M enters the monitoring area P1 c isdetected at a time t3, the time t3 is set as a start timing of thefundamental work operation C and also as an end timing of thefundamental work operation B. Further, when a motion by which the handof the worker M enters the monitoring area Ptd is detected at a time t4,the time t4 is set as a start timing of the fundamental work operation Dand also as an end timing of the fundamental work operation C. Further,when a motion by which the hand of the worker M enters the monitoringarea P1 a is detected at a time t5, the time t5 is set as a start timingof the fundamental work operation A in the (k+1)th cycle and also as anend timing of the fundamental work operation D in the kth cycle.

When a necessary work video image in which the delimitation informationis set is obtained and thus a completion operation is performed (Yes atstep S107 shown in FIG. 6), an abnormal work exclusion is performed atstep S109. In the step S109, on the basis of the delimitationinformation set as described above, a fundamental work operationpresumed to be abnormal is excluded from the determination data.

Specifically, at step S109, a normal range of work time, which isregarded as normal work, is calculated on the basis of the average worktime calculated from the delimitation information which is set for eachfundamental work operation to automatically exclude the fundamental workoperation whose work time is not within the normal range of work time.In the example shown in FIG. 7, since the work time of the fundamentalwork operation B in the n1 cycle is out of the normal range of worktime, the fundamental work operation B in the n1 cycle is excluded fromthe determination data. Specifically, when the width between tnbs, whichrepresents the work time of the fundamental work operation B, is largerthan th, which represents a normal range of work time, the fundamentalwork operation B is excluded from the determination data.

For example, on the basis of the set delimitation information, afundamental work operation immediately before the fundamental workoperation that has been performed in an order different from apredetermined order is automatically excluded. In the presentembodiment, a predetermined order refers to, for example, that thefundamental work operations are performed in the order of thefundamental work operation A, the fundamental work operation B, thefundamental work operation C, and the fundamental work operation D. Inthe example of FIG. 7, the fundamental work operation C in the n2 cycleis performed following the fundamental work operation A, which isdifferent from the predetermined order. In this case, the fundamentalwork operation A in the n2 cycle immediately before the fundamental workoperation C in the n2 cycle is excluded from the determination data.Since the delimitation motion of the fundamental work operation Bactually performed following the fundamental work operation A cannot bedetected for some reason, the work video image of the fundamental workoperation B may be included in the work video image delimited as thefundamental work operation A. Accordingly, the fundamental workoperation A immediately before the fundamental work operation Cperformed in an order different from the predetermined is excluded fromthe determination data.

When the fundamental work operation presumed to be abnormal is excludedas described above, the determination data generation is performed atstep S111 of FIG. 6 to generate the determination data (annotation data)including the delimitation information corresponding to the video imageof the remaining fundamental work operation, which is regarded as normalwork and has not been excluded. The determination data thus generated isstored in the storage unit 12, and transmitted to a higher level deviceor the like via the communication unit 18 as necessary. In addition, thedata related to the fundamental work operation presumed to be abnormalmay also be separately stored as abnormal data in the storage unit 12 asthe data to be used for learning an abnormal behavior or the like.Further, the control unit 11 that performs the abnormal work exclusionand the determination data generation can correspond to an example of a“generation unit.”

As described above, in the work analysis device 10 according to thepresent embodiment, on the basis of a delimitation motion (predeterminedmotion) which is preset for each fundamental work operation,delimitation information is generated to delimitate the work videoacquired by the imaging unit 13 for each fundamental work operation at atiming when the delimitation motion is detected, and determination datais generated to include the work video image acquired by the imagingunit 13 and the set delimitation information.

Accordingly, the work video image can be automatically delimited foreach fundamental work operation by simply acquiring the delimitationmotion that is performed for each fundamental work operation by theworker M. Accordingly, the determination data for determining whether ornot the correct work is performed can be generated in real time on thebasis of the work video image obtained by imaging the predetermined workrepeatedly performed.

In particular, in the present embodiment, a fundamental work operationis assembling a part 20, which has been taken out of the parts box 30associated with the fundamental work operation, to the workpiece(assembly target) W, and the delimitation motion described above is amotion of taking out the part 20 from the parts box 30. Since the workvideo image can be delimited at the detection timing of the motion thatis essential for the fundamental work operation, there is no need offorcing a motion irrelevant to the intended fundamental work operation.Accordingly, in generation of the determination data, a work burden onthe worker M can be reduced.

Further, in the abnormal work exclusion and the determination datageneration described above, a normal range of work time, which isregarded as normal work, is calculated for each fundamental workoperation on the basis of the set delimitation information, and thefundamental work operation whose work time is within the normal range ofwork time is generated as the determination data. Accordingly, the videoimage of the fundamental work operation that is regarded as normal workin view of the work time can be automatically taken as determinationdata, which contributes to improvement in reliability of determinationdata. On the other hand, in the abnormal work exclusion and thedetermination data generation described above, the determination data isgenerated to exclude the fundamental work operation whose work time isout of the normal range of work time. Accordingly, the video image ofthe fundamental work operation that is not regarded as normal work canbe automatically excluded from determination data, which contributes toimprovement in reliability of determination data.

Therefore, the state of work manually performed by the worker, forexample, part assembly work (manual work), can be analyzed in moreefficient and more accurate manner.

Furthermore, in the abnormal work exclusion and the determination datageneration described above, the determination data is generated toexclude a fundamental work operation immediately before the fundamentalwork operation that has been performed in an order different from apredetermined order on the basis of the set delimitation information.Since the video image of the fundamental work operation that is notregarded as normal work is thus automatically excluded fromdetermination data, the reliability of determination data can beimproved.

Second Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a second embodiment will now be described.

The second embodiment mainly differs from the aforementioned firstembodiment in that the delimitation motion is a motion of transferring apart to the assembly position in the assembly target. The componentswhich are substantially the same as those of the first embodiment aredenoted by the same reference signs, and the description thereof will beomitted.

In the present embodiment, the predetermined work to be analyzed is, aswith the first embodiment described above, the work including afundamental work operation A of assembling the part 20 a in the partsbox 30 a to the workpiece W, a fundamental work operation B ofassembling the part 20 b in the parts box 30 b to the workpiece W, afundamental work operation C of assembling the part 20 c in the partsbox 30 c to the workpiece W, and a fundamental work operation D ofassembling the part 20 d in the parts box 30 d to the workpiece W inthis order.

Unlike the above first embodiment, in the imaging range by the imagingunit 13 as shown in FIG. 8, a range corresponding to an assemblyposition where the part 20 a is assembled to the workpiece W is set as amonitoring area P2 a. A range corresponding to an assembly positionwhere the part 20 b is assembled to the workpiece W is set as amonitoring area P2 b. A range corresponding to an assembly positionwhere the part 20 c is assembled to the workpiece W is set as amonitoring area P2 c. A range corresponding to an assembly positionwhere the part 20 d is assembled to the workpiece W is set as amonitoring area P2 d.

As shown in FIG. 9A, a motion that the part 20 a has been transferred tothe monitoring area P2 a is set as the delimitation motion for thefundamental work operation A. As shown in FIG. 9B, a motion that thepart 20 b has been transferred to the monitoring area P2 b is set as thedelimitation motion for the fundamental work operation B. As shown inFIG. 10A, a motion that the part 20 c has been transferred to themonitoring area P2 c is set as the delimitation motion for thefundamental work operation C. As shown in FIG. 10B, a motion that thepart 20 d has been transferred to the monitoring area P2 d is set as thedelimitation motion for the fundamental work operation D. FIGS. 8 to 10illustrate enlarged views of the respective ranges corresponding to thework W in the imaging range by the imaging unit 13. In FIGS. 9 and 10,illustration of the hand of the worker M holding the part 20 is omittedfor convenience.

Similar to the first embodiment, as shown in FIG. 6, the work analysisprocessing is performed by the control unit 11 in the presentembodiment. When any of the above delimitation motions is detected (Yesat S103) while the work video image is being acquired, the delimitationinformation for delimiting the work video image at the detection timingis set (S105). The steps from step S103 onward are repeated until acompletion operation is performed. When a completion operation isperformed (Yes at S107) during this repetition, the determination datais generated to exclude the fundamental work operation presumed to beabnormal (S109, S111).

As described above, in the work analysis device 10 according to thepresent embodiment, a fundamental work operation is assembling a part20, which is associated with the fundamental work operation, to theworkpiece (assembly target) W, and the delimitation motion(predetermined motion) is a motion of transferring the part 20 to anassembly position in the workpiece W. Since the work video image can bedelimited at the detection timing of the motion that is essential forthe fundamental work operation, there is no need of forcing a motionirrelevant to the intended fundamental work operation. Accordingly, ingeneration of the determination data, a work burden on the worker M canbe reduced.

Third Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a third embodiment will now be described.

The third embodiment mainly differs from the first embodiment in theprocess for detecting the delimitation motion. The components which aresubstantially the same as those of the first embodiment are denoted bythe same reference signs, and the description thereof will be omitted.

As shown in FIG. 11, the work analysis device 10 according to thepresent embodiment includes a monitoring device 40 that detects a motionof taking out the part 20 from the parts box 30 as the delimitationmotion. The monitoring device 40 is a device for detecting thedelimitation motion by monitoring a change in capacitance. Themonitoring device 40 includes a sensor circuit 41, six strip-shapedconductors 42 x 1 to 42 x 6, six strip-shaped conductors 42 y 1 to 42 y6, and a work table 43. Further, in FIG. 11 and FIG. 12 described later,the work table 43 is indicated by the dotted and dashed line forconvenience.

The sensor circuit 41 is a circuit that detects a change in capacitance(unit: F) of the respective conductors 42 x 1 to 42 x 6 and therespective conductors 42 y 1 to 42 y 6, and is controlled by the controlunit 11 to output the monitor results to the control unit 11.

As shown in FIG. 11, the respective conductors 42 x 1 to 42 x 6 and therespective conductors 42 y 1 to 42 y 6 are arranged in a matrix pattern.For example, when the hand approaches an area where the conductors 42 x3 and the conductor 42 y 3 overlap each other, the capacitance of theconductor 42 x 3 and the conductor 42 y 3 changes, and the change incapacitance is detected by the sensor circuit 41. That is, theconductors 42 x 1 to 42 x 6 and the conductors 42 y 1 to 42 y 6 form 36areas where the change in capacitance can be detected (hereinafter, alsoreferred to as work areas). The work area where the hand approaches canbe detected by monitoring the change in capacitance in the respectivework areas arranged in a matrix. Each conductor is subjected toinsulation treatment or the like so that another conductor overlappedtherewith has a small change in capacitance.

In the present embodiment, the parts boxes 30 are positioned on the worktable 43, which is disposed to cover the respective work areas, so thatthe sensor circuit 41 can detect a motion of taking out the part 20 fromthe parts box 30 (delimitation motion) and the position of the parts box30 on the basis of the work area where a change in capacitance largerthan a predetermined threshold is detected. That is, the delimitationmotion and the like can be detected by simply performing a normal workoperation without requiring a special motion. Further, the number ofparts boxes 30 used for the work can also be detected on the basis ofthe order of the work areas in which the capacitance changes.

In particular, each work area can be associated with the type of theparts box 30 positioned thereon to specify the part taken out as well asto detect the delimitation motion. For example, as shown in FIG. 12,when the parts boxes 30 a to 30 d are positioned on the work table 43and associated as above, the sensor circuit 41 can detect when theconductors 42 x 3, 42 x 4, 42 y 2, and 42 y 3 have a change incapacitance larger than the other conductors, and thus detect that thepart 20 b is taken out of the parts box 30 b as well as detect thedelimitation motion.

As described above, according to the present embodiment, acharacteristic configuration for monitoring the change in capacitance inthe respective conductors 42 x 1 to 42 x 6 and the respective conductors42 y 1 to 42 y 6 enables detection of the delimitation motion and thetype of the part 20 taken out, and the like. This characteristicconfiguration can also be applied to other embodiments and the like. Theabove work areas are not limited to the conductors 42 x 1 to 42 x 6 andthe conductors 42 y 1 to 42 y 6, and other members capable of monitoringa change in capacitance can also constitute the work areas. For example,a configuration may be adopted in which one conductor capable ofmonitoring a change in capacitance is disposed in each work area.Further, the number of work areas is not limited to 36 (6×6), and mayvary depending on the arrangement of the parts boxes 30. Theconfiguration for detecting the delimitation motion and the like bymonitoring a change in capacitance in the conductor is applied todetection of the delimitation motion of taking out the part 20 from theparts box 30 as described above, but not limited thereto. For example,the configuration may also be applied to detection of the delimitationmotion of transferring the part 20 to the assembly position in theassembly target.

Fourth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a fourth embodiment will now be described.

The fourth embodiment mainly differs from the first embodiment in theprocess for setting the monitoring area. The components which aresubstantially the same as those of the first embodiment are denoted bythe same reference signs, and the description thereof will be omitted.

As shown in FIG. 13A, the work analysis device 10 according to thepresent embodiment includes a monitoring device 40 a that detects themonitoring area described above, that is, a position where the parts box30 is located. The monitoring device 40 a includes a plurality ofpiezoelectric switches 44 arranged in a line at equal intervals withoutsubstantially no gap on the surface of the shelf 2 on which the partsbox 30 is to be positioned. The piezoelectric switches 44 are configuredto output a signal corresponding to the pressure applied to the controlunit 11. The control unit 11 can detect which piezoelectric switch 44 isin a pressed state.

Specifically, as shown in FIG. 13B, when the parts box 30 is positionedon a part of an observation area, which is composed of the surfaces ofthe respective piezoelectric switches 44 to be pressed, a signal isoutputted only from the piezoelectric switch 44 on which the parts box30 is positioned (see reference numeral 44 a in FIG. 13B). Accordingly,a position on which the parts box 30 is positioned (monitoring area) inthe observation area can be detected by monitoring the pressed state ofthe respective piezoelectric switches 44 by the control unit 11. In FIG.13B, the parts box 30 is indicated by the dotted line for convenience.

Instead of the piezoelectric switch 44, a physical switch such as acontact switch that can detect when a part of the parts box contacts thedetection surface may also be used to detect a position on which theparts box is positioned (monitoring area) in the observation area.

In the environment that the frequency of usage is high rather thannon-usage environment, a modified example of the present embodiment, asshown in FIG. 14A, may also be adopted in which a monitoring device 40 bis used instead of the monitoring device 40 a. In the environment inwhich the monitoring device 40 b according to the modified example isused, at least the back surface of the parts box 30 has conductivity.The monitoring device 40 b is configured to detect a change inconductive state in the observation area to thereby detect a positionwhere the parts box 30 is located (monitoring area). The monitoringdevice 40 b includes a plurality of first conductors 45 and secondconductors 46, which are of the same number and arranged parallel toeach other, on the surface on which the parts box 30 is to bepositioned. As shown in FIG. 14B, the monitoring device 40 b can detectthe first conductor 45 and the second conductor 46 in an electricallyconductive state to thereby detect the range occupied by the conductors45 and 46 thus detected (see reference numerals 45 a and 46 a in FIG.14B) as a position where the parts box 30 is located. In FIG. 14B, theparts box 30 is indicated by the dotted line for convenience.

In particular, the resistance of the conductor provided on the rearsurface of the parts box 30 may be different for each parts box 30 sothat the resistance detected via the first conductor 45 and the secondconductor 46 can be used to determine the type of the parts box 30positioned thereon.

The characteristic configuration of the present embodiment for detectingeach monitoring area by using the piezoelectric switch 44 or the likeand the characteristic configurations of the modified examples of thepresent embodiment for detecting each monitoring area by using the firstconductor 45 and the second conductor 46 or the like can also be appliedto other embodiments.

Fifth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a fifth embodiment will now be described.

The fifth embodiment mainly differs from the first embodiment in theprocess for setting the monitoring area and the process for detectingthe delimitation motion with the height of the hand taken intoconsideration. The components which are substantially the same as thoseof the first embodiment are denoted by the same reference signs, and thedescription thereof will be omitted.

In the present embodiment, prior to the work analysis processing,monitoring area setting is performed by the control unit 11 to set eachmonitoring area. In the monitoring area setting, a monitoring areacorresponding to a parts box can be set by imaging the trajectory of thefinger tracing an upper end face 31 of the peripheral wall of the partsbox 30 by using the imaging unit 13. For example, as illustrated in FIG.15, the monitoring area Plc corresponding to the parts box 30 c can beset by imaging the trajectory of the finger tracing the upper end face31 of the peripheral wall of the parts box 30 c by using the imagingunit 13.

In the work analysis processing, the delimitation motion that the handof the worker M enters the monitoring area is detected taking the heightof the hand of the worker M into consideration. For example, when thehand, intending to pick up a part contained in the parts box 30 b,passes over the parts box 30 a, which is adjacent to the parts box 30 b,it may be erroneously recognized that the hand enters the monitoringarea corresponding to the parts box 30 a, which causes erroneousdetection of the delimitation motion.

For this reason, in the present embodiment, a ToF (Time-of-Flight)camera that can measure a distance to an object to be imaged is used asthe imaging unit 13. As shown in FIG. 16, the height of the handentering the monitoring area from a placement surface 2 a is measured onthe basis of the placement surface 2 a of the shelf 2 on which the partsboxes 30 are placed. When the height of the hand entering the monitoringarea from the placement surface 2 a is larger than a predeterminedthreshold h1, the detection is invalid and the motion is not determinedas the delimitation motion. On the other hand, when the height of thehand entering the monitoring area from the placement surface 2 a is notlarger than the predetermined threshold h1, the detection is valid andthe motion is determined as the delimitation motion. Further, anotherToF camera may be provided separately from the imaging unit 13 tomeasure the height of the hand entering the monitoring area from theplacement surface 2 a by using this camera.

Accordingly, for example, even if the hand intending to pick up a partcontained in the parts box 30 b enters the monitoring area correspondingto the parts box 30 a, the detection is valid when the height of thehand from the placement surface 2 a is larger than the predeterminedthreshold h1. Thus, erroneous detection of the delimitation motion canbe prevented.

Further, the detection can be determined to be invalid on the basis ofnot only the height of the hand from the placement surface 2 a, but alsothe height of the hand from the upper end face 31 of the parts box 30(see a threshold h2 in FIG. 16). The characteristic configuration of thepresent embodiment for determining whether the detection is invalid ornot on the basis of the setting of the monitoring area by using thetrajectory of the finger tracing as described above or the height of thehand can also be applied to other embodiments and the like.

Sixth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a sixth embodiment will now be described.

The sixth embodiment mainly differs from the first embodiment in thatthe delimitation motion is detected by using a change in weight. Thecomponents which are substantially the same as those of the firstembodiment are denoted by the same reference signs, and the descriptionthereof will be omitted.

As shown in FIGS. 17A and 17B, the work analysis device 10 according tothe present embodiment includes a monitoring device 40 c that detects amotion of taking out the part 20 from the parts box 30 and a motion ofassembling the part 20 to the assembly target as the delimitationmotion. The monitoring device 40 c is a device that detects thedelimitation motion by monitoring a change in weight of each parts box30 and the assembly target, and includes a weight sensor 47 that canmeasure the weight of each parts box 30 and the assembly target. Theweight sensor 47 has a weighing surface formed as a flat surface so thatthe parts boxes 30 and the assembly target can be placed thereon at thesame time, and is assembled to the shelf 2.

Referring to the drawings, the work analysis processing performed in thepresent embodiment will now be described. As shown in FIGS. 17A and 17B,in the work described below in detail, four parts boxes 30 a to 30 d andthe workpiece W to which the parts 20 a to 20 d are assembled are placedon the weighing surface of the weight sensor 47. In the presentembodiment, the predetermined work to be analyzed is the work includinga fundamental work operation A of assembling the part 20 a in the partsbox 30 a to the workpiece W, a fundamental work operation B ofassembling the part 20 b in the parts box 30 b to the workpiece W, afundamental work operation C of assembling the part 20 c in the partsbox 30 c to the workpiece W, and a fundamental work operation D ofassembling the part 20 d in the parts box 30 d to the workpiece W inthis order.

At the start of assembly work immediately after the workpiece W on whichthe parts 20 a to 20 d are not assembled is placed on the weighingsurface, a measured value F measured by the weight sensor 47 is Fo. Asshown in FIG. 17A, when the part 20 a is taken out of the parts box 30 afor starting the fundamental work operation A, a value measured by theweight sensor 47 decreases from Fo to Fa. The measured value Fa is avalue obtained by subtracting the weight of the part 20 a from themeasured value Fo. Accordingly, when the value measured by the weightsensor 47 decreases from Fo to Fa (see time t1 in FIG. 18), thedelimitation motion of taking out the part 20 a from the parts box 30 acan be detected. Subsequently, as shown in FIG. 17B, as the part 20 a isassembled to the workpiece W, the value measured by the weight sensor 47increases from Fa to Fo.

Accordingly, when the value measured by the weight sensor 47 returns(increases) from Fa to Fo (see time t2 in FIG. 18), the delimitationmotion of assembling the part 20 a to the workpiece W can be detected.Subsequently, when the value measured by the weight sensor 47 decreasesfrom Fo to Fb (a value obtained by subtracting the weight of the part 20b from the measured value Fo) (see time t3 in FIG. 18), the delimitationmotion of taking out the part 20 b from the parts box 30 b can bedetected. Further, when the value measured by the weight sensor 47returns from Fb to Fo (see time t4 in FIG. 18), the delimitation motionof assembling the part 20 b to the workpiece W can be detected.Subsequently, when the value measured by the weight sensor 47 decreasesfrom Fo to Fc (a value obtained by subtracting the weight of the part 20c from the measured value Fo) (see time t5 in FIG. 18), the delimitationmotion of taking out the part 20 c from the parts box 30 c can bedetected. Further, when the value measured by the weight sensor 47returns from Fc to Fo (see time t6 in FIG. 18), the delimitation motionof assembling the part 20 c to the workpiece W can be detected.

Subsequently, when the value measured by the weight sensor 47 decreasesfrom Fo to Fd (a value obtained by subtracting the weight of the part 20d from the measured value Fo) (see time t7 in FIG. 18), the delimitationmotion of taking out the part 20 d from the parts box 30 d can bedetected. Further, when the value measured by the weight sensor 47returns from Fd to Fo (see time t8 in FIG. 18), the delimitation motionof assembling the part 20 d to the workpiece W can be detected.

Thus, in the present embodiment, the delimitation motion can becorrectly detected by using a change in weight measured by the weightsensor 47. Accordingly, for example, even if an unintended part iserroneously picked up from a different parts box, a large change inweight as in the time of assembly can be prevented from being detectedwhen the part is returned to the original parts box. Thus, erroneousdetection of the delimitation motion due to an error in picking up ofthe part can be reduced. Further, since a change in weight due tofalling of a part is different from a change in weight during assembly,it is possible to reliably distinguish completion of assembly from merefalling of a part on the basis of the change in weight detected even ifa part falls onto the assembly target during assembly.

The weight sensor 47 may be a non-contact capacitance sensor, acapacitance weight sensor, a thin film pressure sensor, or the like.Further, the monitoring device 40 c is not limited to use a singleweight sensor 47 to measure the weight of the parts boxes 30 and theassembly target, may also use two or more weight sensors to measure theweight of the parts boxes 30 and the assembly target. The characteristicconfiguration of the present embodiment for detecting the delimitationmotion by using a change in weight measured by the weight sensor canalso be applied to other embodiments and the like.

Seventh Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a seventh embodiment will now bedescribed.

The seventh embodiment differs from the first embodiment in the processfor setting the monitoring area. The components which are substantiallythe same as those of the first embodiment are denoted by the samereference signs, and the description thereof will be omitted.

In the work analysis device 10 according to the present embodiment, thetrajectory of the hand of the worker M when taking out the part 20 fromthe parts box 30 and the trajectory of the hand of the worker M whenassembling the part 20 to the assembly target are monitored to set themonitoring area on the basis of the monitoring result.

Specifically, the hand of the worker M is recognized from the videoimage acquired by the imaging unit 13, and the moving direction and themoving distance of the hand imaged are detected at a predetermined timeinterval (for example, frame interval). When the state in which themoving distance is equal to or less than a predetermined value continuesat least for a predetermined period, it is determined as the state inwhich the hand is retained (hereinafter, also referred to as a retainedstate), and a temporary monitoring area is set on the basis of the handimaged at the time of the retained state. This is because a motion ofthe hand taking out the part 20 from the parts box 30 and a motion ofthe hand assembling the part 20 to the assembly target are likely to bein the retained state.

For example, as shown in FIG. 19, when the hand, having assembled thepart 20 a to the workpiece W, moves toward the parts box 30 b to pick upthe part 20 b therefrom, the hand is in the retained state when it istaking out the part 20 b from the parts box 30 b. Accordingly, atemporary monitoring area (see reference numeral Po1 b in FIG. 19) forthe parts box 30 b is set on the basis of the hand in the retainedstate. Subsequently, as shown in FIG. 20, when the hand, having takenout the part 20 b, moves toward the assembly position in the workpieceW, the hand is in the retained state when assembling the part 20 b tothe assembly position in the workpiece W. Accordingly, a temporarymonitoring area (see reference numeral Po2 b in FIG. 20) for theassembly position of the part 20 b is set on the basis of the hand inthe retained state.

In the present embodiment, the temporary monitoring area is set as arectangular area containing the imaging range of the hand in theretained state as the temporary monitoring area Po1 b in FIG. 19 and thetemporary monitoring area Po2 b in FIG. 20. However, this is merely anexample, and, for example, the temporary monitoring area may also be setas a circle area or an oval area circumscribed about the imaging rangeof the hand in the retained state or a circle area or an oval areainscribed in the imaging range of the hand in the retained state.

By repeating each fundamental work operation, the temporary monitoringarea for each parts box 30 and the temporary monitoring area for eachassembly position are sequentially stored in the storage unit 12. Afterthat, in order to improve estimation accuracy for the monitoring areaand optimize the monitoring area, an overlapping area for each temporarymonitoring area is obtained and weighted so that the monitoring area isset according to the weighting.

For example, when a temporary monitoring area is set 100 times for onemonitoring area, an area overlapping 50 times or more is set to “3,” anarea overlapping 20 times or more is set to “2,” and an area overlapping10 times or more is set to “1.” When the monitoring area is set toinclude all the temporary monitoring areas weighted with “3,” themonitoring area (see the hatched area) can be set as shown in FIG. 21A.Further, when the monitoring area is set to include all the temporarymonitoring areas weighted with “2” or more, the monitoring area (see thehatched area) can be set as shown in FIG. 21B.

The temporary monitoring area may also be set for one monitoring area asa predetermined shape such as a rectangle whose center is a middleposition between the center position of the hand in the previousretained state and the center position of the hand in the currentretained state. The characteristic configuration of the presentembodiment for setting one monitoring area by using a plurality oftemporary monitoring area as described above can also be applied toother embodiments and the like.

Eighth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to an eighth embodiment will now bedescribed.

The eighth embodiment differs from the first embodiment in that theprocess of setting a monitoring area corresponding to each parts box isperformed by using information code provided on each parts box. Thecomponents which are substantially the same as those of the firstembodiment are denoted by the same reference signs, and the descriptionthereof will be omitted.

In the present embodiment, as shown in FIG. 22, an information code forpositional detection (hereinafter, also referred to as a parts box codeCa) is provided on each parts box 30. The work analysis device 10includes a second imaging unit 13 a that images the parts box code Ca.The second imaging unit 13 a functions as a camera for setting amonitoring area, has the same function as the imaging unit 13, and isconfigured to store the captured image in the storage unit 12 so as tobe analyzed by the control unit 11. The second imaging unit 13 a isprovided at a position within the imaging range of the imaging unit 13and the front face of the second imaging unit 13 a as viewed from theworker M is configured to image the parts box code Ca on each parts box30 disposed on the shelf 2. An information code for calculating arelative positional relationship between the second imaging unit 13 aand the imaging unit 13 (hereinafter, also referred to as a camera codeCb) is provided on an outer surface (top) of the second imaging unit 13a, which is oriented to the imaging unit 13. In the present embodiment,the parts box code Ca and the camera code Cb are implemented as a QRcode (registered trademark). However, this is merely an example, and,for example, a one-dimensional code such as a bar code and other typesof second code may also be used.

In the present embodiment, prior to the work analysis processing,monitoring area setting is performed by the control unit 11 to set eachmonitoring area. In the monitoring area setting, the relativecoordinates of the parts box 30 relative to the second imaging unit 13 ais calculated from an image of the parts box code Ca captured by thesecond imaging unit 13 a. As the relative coordinates of the secondimaging unit 13 a relative to the imaging unit 13 is calculated from animage of the camera code Cb captured by the imaging unit 13, therelative coordinates of the parts box 30 relative to the imaging unit 13is estimated, and the monitoring area for each parts box 30 is set onthe basis of the estimation result.

Accordingly, the parts box code Ca includes information on the part 20contained in the parts box 30 to which the parts box code Ca is attachedas well as information on the size of the parts box 30 such as height,width, and depth (hereinafter, also referred to as a box size), the sizeindicating the dimensions, shape, cell number, and the like of the partsbox code Ca (hereinafter, also referred to as a code size), and theattached position of the parts box code Ca to the parts box 30(hereinafter, also referred to as a code position).

Further, the camera code Cb includes information for calculating therelative positional relationship between the imaging unit 13 and thesecond imaging unit 13 a from the captured image of the camera code Cb,such as the attached position of the camera code Cb to the secondimaging unit 13 a and the code size.

Referring to the drawings, the monitoring area setting in the presentembodiment will now be specifically described. In the followingdescription, the case where one parts box 30 to which the parts box codeCa is provided is disposed on the shelf 2 will be described in detail.

Prior to the work analysis processing, the monitoring area setting isinitiated by the control unit 11 in response to a predeterminedoperation. At step S201 in FIG. 23a , parts box relative coordinateestimation is performed to estimate the relative coordinates of theparts box 30 relative to the second imaging unit 13 a. In the subroutineof step S201, as shown in FIG. 24, the second imaging unit 13 a is readyto image the parts box code Ca (S211 in FIG. 24), and then decoding forreading the captured parts box code Ca is performed (S213).

When the parts box code Ca is successfully read since the parts box 30is disposed on the shelf 2 (Yes at S215), the box size, code size, codeposition, and the like are obtained. Then, reading angle calculation isperformed at step S217. At step S217, an angle of the parts box code Carelative to the second imaging unit 13 a is calculated as a readingangle α on the basis of the range of the parts box code Ca in the imagecaptured by the second imaging unit 13 a.

For example, the parts box 30 is disposed inclined relative to thesecond imaging unit 13 a as shown in FIG. 26A. Accordingly, when theparts box code Ca is captured as the image shown in FIG. 26B, thereading angle α of the parts box code Ca relative to the second imagingunit 13 a is calculated on the basis of the length ratio of four sidesobtained from four corner positions of the parts box code Ca (see FIG.25A).

Subsequently, reading distance calculation is performed at step S219 inFIG. 24. At step S219, a distance from the second imaging unit 13 a tothe parts box code Ca is calculated as a reading distance y on the basisof the number of pixels of the parts box code Ca in the image capturedby the second imaging unit 13 a. The reading distance y can becalculated on the basis of the image corrected so that each angle offour corners of the parts box code Ca becomes 90° by using the readingangle α calculated as described above.

For example, for the parts box code Ca captured as shown in FIG. 27, therelationship of equation (1) holds when the number of pixels of theentire captured image in the x direction (horizontal direction in FIG.27) is x1, the number of pixels of the parts box code Ca in the xdirection is x2, the actual size corresponding to the entire capturedimage in the x direction at the position of the parts box code Ca is x3,and the actual size of the actual parts box code Ca in the x directionis x4. Further, for the reading distance y, the relationship of equation(2) is established from the size x3 corresponding to the entire capturedimage in the x direction at a position of the parts box code Ca, and anangle θ obtained from the angle of view, the resolution, and the like ofthe second imaging unit 13 a.

x1:x2=x3:x4  (1)

y=x3/2×tan θ  (2)

Since x1, x4, and θ are known, the reading distance y can be calculatedby the following equation (3) obtained from the equations (1) and (2) onthe basis of the number of pixels x2 of the parts box code Ca in the xdirection.

y=(x1×x4/x2)/2×tan θ  (3)

When the reading angle α and the reading distance y are calculated asdescribed above, relative coordinate estimation is performed at stepS221 in FIG. 24. At step S221, first, the relative coordinates of theparts box code Ca relative to the second imaging unit 13 a is calculatedon the basis of the reading angle α and the reading distance ycalculated as described above. Then, the relative coordinates of theparts box 30 relative to the second imaging unit 13 a is estimated onthe basis of the relative coordinates of the parts box code Ca thuscalculated, the code position, code size, box size, and the like readfrom the parts box code Ca. Thus, how the parts box 30 is positionedrelative to the second imaging unit 13 a can be estimated.

When the relative coordinates of the parts box 30 relative to the secondimaging unit 13 a is estimated as described above, and thus parts boxrelative coordinate estimation at step S201 in FIG. 23 is completed, therelative coordinates of the second imaging unit 13 a relative to theimaging unit 13 is estimated in camera relative coordinate estimation atstep S203. In the subroutine of step S203, as shown in FIG. 25, theimaging unit 13 is ready to image the camera code Cb (S231 in FIG. 25),and then decoding for reading the captured camera code Cb is performed(S233).

When the camera code Cb is successfully read (Yes at S235), readingangle calculation is performed at step S237, and the angle of the cameracode Cb relative to the imaging unit 13 is calculated as a reading angleby using the same calculation method as the reading angle calculation ofstep S217 in FIG. 24. After step S237 in FIG. 25, reading distancecalculation is performed at step S239, and the distance from the imagingunit 13 to the camera code Cb is calculated as a reading distance on thebasis of the number of pixels of the camera code Cb in the imagecaptured by the imaging unit 13 by using the same calculation method asthe reading distance calculation of step S219 in FIG. 24.

When the reading angle and the reading distance are calculated asdescribed above, relative coordinate estimation is performed at stepS241 in FIG. 25, and the relative coordinates of the camera code Cbrelative to the imaging unit 13 is calculated by using the samecalculation method as the relative coordinate estimation of step S221 inFIG. 24. Then, the relative coordinates of the second imaging unit 13 arelative to the imaging unit 13 is estimated on the basis of therelative coordinates of the camera code Cb thus calculated, the readingresults read from the camera code Cb. Thus, how the second imaging unit13 a is positioned relative to the imaging unit 13 can be estimated.When the second imaging unit 13 a is disposed at a predeterminedposition relative to the imaging unit 13, the camera relative coordinateestimation described above may be omitted since the relative coordinatesof the second imaging unit 13 a relative to the imaging unit 13 arespecified.

When the relative coordinates of the second imaging unit 13 a relativeto the imaging unit 13 is estimated as described above, and thus camerarelative coordinate estimation at step S203 in FIG. 23 is completed, theparts box position calculation is performed at step S205. At step S205,a position (relative coordinates) of the parts box 30 relative to theimaging unit 13 is calculated on the basis of the relative coordinatesof the parts box 30 relative to the second imaging unit 13 a and therelative coordinates of the second imaging unit 13 a relative to theimaging unit 13.

Accordingly, since the area occupied by the parts box 30 in the imagecaptured by the imaging unit 13 can be estimated, the area estimated asthe parts box 30 in the captured image is set as a monitoring area inthe setting at step S207, and the monitoring area setting ends. When aplurality of parts box codes Ca are imaged by the second imaging unit 13a in a decodable manner, the monitoring area can be set for each partsbox code Ca in the order of execution described above. In particular,since the parts box code Ca also includes the information of the part20, the type of the part 20 corresponding to the monitoring area canalso be specified.

In the present embodiment, as shown in FIG. 28A, the parts box code Cais provided close to one side of a side face of the parts box 30 whichis oriented to the second imaging unit 13 a, and has a size as large aspossible. However, this is merely an example, and the parts box code Camay also be provided at any position as long as it corresponds to aposition of a code recorded. For example, as shown in FIG. 28B, theparts box code Ca may be provided at the center of a side face of theparts box 30 which is oriented to the second imaging unit 13 a, or asshown in FIG. 28C, may be provided close to one side of a side face, andhas a relatively small size.

Further, assuming the case where the position on which the parts boxcode Ca is provided in parts box 30 cannot be located, it is alsopossible to calculate the relative coordinates of the parts box code Caand the surface of the parts box 30 on which the parts box code Ca isprovided from the image captured by the second imaging unit 13 a andestimate the relative coordinates of the parts box 30 relative to thesecond imaging unit 13 a by using the calculation result and the boxsize and code size. In this case, the code position is not recorded inthe parts box code Ca.

For example, when the parts box code Ca is provided on the parts box 30as shown in FIG. 29A, and thus the parts box code Ca and the parts box30 are imaged as shown in FIG. 29B, the above relative coordinates canbe calculated by detecting where on the surface of the parts box 30 thecomponent box code Ca is provided and how it is skewed by using imageprocessing such as edge search. For example, as indicated by the boldline in FIG. 29B, the length (known) and the angle of the upper end faceof the parts box 30 can be compared with the length (known) and theangle of one side of the parts box code Ca to thereby detect where onthe surface of the parts box 30 the parts box code Ca is provided andhow it is skewed.

Although the front face of the second imaging unit 13 a as viewed fromthe worker M is configured to image the parts box code Ca on each partsbox 30 disposed on the shelf 2, the back face, for example, of thesecond imaging unit 13 a as viewed from the worker M may also beconfigured to capture images.

FIG. 30 illustrate a first modified example of the present embodiment,in which, under the assumption that the parts boxes 30, each having theparts box code Ca provided on the bottom, are placed on a transparentportion of the shelf 2, the second imaging unit 13 a is disposed at aposition where it can image the parts box code Ca from underside via thetransparent portion of the shelf 2. In this case, for example, theinformation codes Cc are provided at predetermined positions surroundinga transparent portion of the shelf 2 in the imaging range of the secondimaging unit 13 a. Since information on the position of the shelf 2 onwhich the information code Cc is provided and the like is recorded inthe information code Cc, the relative coordinates of the shelf 2relative to the second imaging unit 13 a can be estimated from the imagecaptured by the second imaging unit 13 a. Accordingly, by estimating therelative coordinates of the shelf 2 relative to the imaging unit 13 byusing another information code or the like, the relative coordinates ofthe second imaging unit 13 a relative to the imaging unit 13 can beestimated. Thus, on the basis of the relative coordinates of the partsbox 30 relative to the second imaging unit 13 a and the relativecoordinates of the second imaging unit 13 a relative to the imaging unit13, the position of the parts box 30 relative to the imaging unit 13 canbe calculated. Therefore, the area occupied by the parts box 30 in theimage captured by the imaging unit 13 can be set as a monitoring area.

In particular, since the parts box code Ca is provided on the bottom ofthe parts box 30, the parts box code Ca can be increased in sizecompared with the case where the parts box code Ca is provided on theside face or the like of the parts box 30. Accordingly, the accuracy incoordinate estimation by reading the parts box code Ca can be improved.Further, since the parts box code Ca and the information code Cc can beimaged at the same time by the second imaging unit 13 a, the positionalrelationship between the parts box code Ca and the information code Cccan also be calculated with improved accuracy. Further, since the secondimaging unit 13 a can be disposed under the shelf 2, a space-saving workanalysis device 10 can be achieved even if it includes the secondimaging unit 13 a. In addition, the parts box code Ca can also beprovided on the parts box, such as a pallet, having no space on the sideface or on the top for providing the information code. In aconfiguration in which the front face or back face of the second imagingunit 13 a as viewed from the worker M is configured to image the partsbox 30, the parts box codes Ca can be imaged when the parts boxes 30 arearranged in one array. Further, in a configuration in which the secondimaging unit 13 a is configured to image the parts box 30 fromunderside, the parts box codes Ca can be imaged even when the partsboxes 30 are arranged in two or more arrays.

Further, a transparent portion of the shelf 2 is not limited to arectangular shape, and may also has a circular shape, for example, as inthe second modified example of the present embodiment shown in FIG. 31.Further, as shown in FIG. 31, a markers Cm or the like provided at aspecific position may also be used instead of the information code Cc.When the second imaging unit 13 a is disposed at a predeterminedposition relative to the shelf 2, the information code Cc may be omittedsince the relative coordinates of the shelf 2 relative to the secondimaging unit 13 a are specified.

In addition, the parts box code Ca is not limited to being provided onthe side face or on the bottom of the parts box 30, and, for example,when the parts box 30 has a thick peripheral wall as shown in FIG. 32A,the parts box code Ca may also be provided on each corner or the like onthe upper end face of the peripheral wall. Further, the parts box codeCa provided on the upper end face of the peripheral wall may beone-dimensional code in view of readability. For example, when the partsbox 30 has an upper lid 32 as shown in FIG. 32B, the parts box code Camay be provided on the upper lid 32.

The characteristic configuration of the present embodiment for setting amonitoring area corresponding to each parts box 30 by using the partsbox code Ca or the like provided on each parts box 30 can also beapplied to other embodiments and the like.

Ninth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a ninth embodiment will now be described.

The ninth embodiment differs from the first embodiment in the processfor setting the monitoring area. The components which are substantiallythe same as those of the first embodiment are denoted by the samereference signs, and the description thereof will be omitted.

In the present embodiment, prior to the work analysis processing,monitoring area setting is performed by the control unit 11 to set eachmonitoring area. In this monitoring area setting, a monitoring areacorresponding to a parts box 30 is set by detecting a movement of theparts box 30 when it is placed on the shelf 2. Specifically, as shown inFIG. 33, the parts box 30 is brought closer to the imaging unit 13, andis then gradually moved away from the imaging unit 13 to be placed onthe shelf 2. In so doing, a range where the distance from the imagingunit 13 gradually changes in the image captured by the imaging unit 13is set as a monitoring area corresponding to the parts box 30.

In the present embodiment, a ToF camera that can measure a distance toan object to be imaged is used as the imaging unit 13. For example, whena ToF camera used as the imaging unit 13 is located closer to an objectto be imaged, the image appears lighter, whereas, when a ToF camera islocated farther away from an object to be imaged, the image appearsdarker. Hence, when the parts box 30 located relatively close to theimaging unit 13 is imaged, the image of the parts box 30 appears inlighter color, as illustrated as a distance image in FIG. 34A. When theparts box 30 is moved farther away from the imaging unit 13, the imageof the parts box 30 appears darker and gets closer to the colorcorresponding to the shelf 2, as illustrated as a distance image in FIG.34B. Further, another ToF camera may be provided separately from theimaging unit 13 to set a monitoring area corresponding to the parts box30 by using the imaging result of this ToF camera.

Referring to the drawings, the monitoring area setting performed by thecontrol unit 11 in the present embodiment will now be specificallydescribed.

Prior to the work analysis processing, the monitoring area setting isinitiated by the control unit 11 in response to a predeterminedoperation. In imaging at step S301 in FIG. 35, the parts box 30 can beimaged while it is being placed on the shelf 2. Subsequently, distancemonitoring is performed at step S303 so that a distance from the imagingunit 13 can be measured according to a captured image in the imagingrange of a portion of the shelf 2 where the parts box 30 is to beplaced.

Subsequently, in determination at step S305, it is determined whether ornot there is an area in which a distance difference is generated due toa movement of an object (hereinafter, also referred to as a distancedifference area) in an imaging range of the imaging unit 13. If theparts box 30 is not brought closer to the shelf 2, and there is nomovement in the imaging range of the imaging unit 13, it is determinedas “No” at step S305 repeatedly.

When a distance difference area is generated due to the parts box 30entering the imaging range of the imaging unit 13 since it is broughtcloser to the imaging unit 13 by the worker M, it is determined as “Yes”at step S305. Then, in determination at step S307, it is determinedwhether or not the distance difference area is attributed to movementaway from the imaging unit 13. As the parts box 30, which has enteredthe imaging range of the imaging unit 13, is moved and placed on theshelf 2 by the worker M, the distance between the imaging unit 13 andthe parts box 30 increases. In this case, the distance difference areabecomes smaller in size while getting darker. Accordingly, it isdetermined that the distance difference area is attributed to movementaway from the imaging unit 13 (Yes at S307).

In this case, in follow area monitoring at step S309, the distancedifference area is set as a follow area corresponding to the parts box30, which is to be followed, and how the follow area changes ismonitored. Subsequently, in determination at step S311, it is determinedwhether or not the follow area defined as described above does notchange any more, that is, whether or not the follow area is unchanged.If the worker M is in the process of placing the parts box 30 on theshelf 2 and thus the parts box 30 is being farther away from the imagingunit 13, the follow area continues to change. In this case, it isdetermined as “No” at step S311 repeatedly.

When the parts box 30 is placed on the shelf 2 by the worker M, thedistance from the parts box 30 to the imaging unit 13, and thus thefollow area, does not change any more. Since the follow area isunchanged, it is determined as “Yes” at step S311. When it is determinedas “Yes” at step S311, the follow area which does not change any more isset as a monitoring area in setting at step S313, and the monitoringarea setting ends. Further, when work is performed by using a pluralityof parts boxes 30, monitoring areas corresponding to the respectiveparts boxes 30 can be set by performing the above monitoring areasetting each time when one parts box 30 is placed on the shelf 2.

According to the present embodiment, since a monitoring area can be setby a general motion of the worker M placing the parts box 30 on theshelf 2 from above, there is no need of providing a device for setting amonitoring area and performing presetting or complicated terminaloperations. Accordingly, it is possible to reduce a motion of the workerM required to set a monitoring area. A monitoring area can be set at atiming when the parts box 30 is placed on the shelf 2. In particular,since the monitoring area is not specified by the worker M, a deviationin setting of the monitoring area can also be reduced. Further, in theabove monitoring area setting, even if an object different from theparts box 30 is present in the imaging range of the imaging unit 13, amonitoring area is not set until the object moves away from the imagingunit 13 and then it stops moving. Accordingly, setting accuracy for themonitoring area can be improved. Further, even if the image or shape ofthe parts box 30 varies, a monitoring area is not set until a followarea is set and then the follow area stops changing. Accordingly, amonitoring area can be set for various types of parts boxes 30.

In addition, the imaging unit 13 is not limited to a ToF camera, and mayalso be a general RGB camera or the like. From the image captured bythis camera, a state in which the parts box 30 is moving away can bedetected to thereby set the above follow area. In this configuration,the farther the parts box 30 moves away from the imaging unit 13, thesmaller the area of the parts box 30 in the captured image. Accordingly,the fact that the parts box 30 is away from the imaging unit 13 can berecognized in a pseudo manner.

The above monitoring area setting is initiated not only by apredetermined operation performed by the worker M, but also, forexample, by capturing predetermined information code by using theimaging unit 13, or by the worker M performing a predetermined gesturein the imaging range of the imaging unit 13.

The characteristic configuration of the present embodiment for setting amonitoring area corresponding to each parts box 30 by detecting themovement of the parts box 30 when it is being placed on the shelf 2 canalso be applied to other embodiments and the like.

Tenth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a tenth embodiment will now be described.

The tenth embodiment differs from the first embodiment in the processfor setting the monitoring area. The components which are substantiallythe same as those of the first embodiment are denoted by the samereference signs, and the description thereof will be omitted.

Although the parts 20 are accommodated in the parts box 30 in variousmanners such as being organized or random, the same type of parts 20 arebasically accommodated together. That is, it is considered that similarparts are together contained in each parts box 30.

In the present embodiment, the image of the parts box 30 divided intoblocks, and the feature of each block is calculated. Then, it isdetermined that the parts 20 are present in a portion where blocks withsimilar features are collected. Various features including an edgefeature such as color histogram or HoG (Histogram of Gradient), and afrequency may be used.

For example, as shown in FIG. 36A, when two parts boxes 30 e and 30 fare placed on the shelf 2, the captured image of the parts boxes 30 eand 30 f is divided into blocks as shown in FIG. 36B, and apredetermined feature is extracted from each block. In the presentembodiment, a feature based on frequency is extracted, and, for example,frequency features shown in FIG. 37 are extracted from the respectiveblocks B1 to B4 in FIG. 36B.

When features are extracted from the respective blocks, the blocks withsimilar features are grouped together. This grouping can be performed,for example, by technique of grouping similar parameters such as k-meansor x-means together into clusters and dividing them.

Then, if there are blocks grouped together in a rectangular area such asthe parts box 30, an area formed of the grouped blocks can be set as amonitoring area corresponding to the parts box 30.

Further, the characteristic configuration of the present embodiment forsetting a monitoring area corresponding to the parts box 30 by using afeature extracted from each block can also be applied to otherembodiments and the like.

Eleventh Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to an eleventh embodiment will now bedescribed.

The eleventh embodiment mainly differs from the first embodiment in theprocess for setting the monitoring area. The components which aresubstantially the same as those of the first embodiment are denoted bythe same reference signs, and the description thereof will be omitted.

Usually, the parts box 30 has a box shape having an upper opening of arectangular shape. The box shape has a characteristic that the upper endface of the peripheral wall is located at the same height. Therefore, inthe present embodiment, a ToF camera is used as the imaging unit 13 tomeasure a distance to an area where the parts box 30 is to be placed.Then, a region located at the same height is extracted, and themonitoring area corresponding to the parts box 30 is set according tothe extracted shape. That is, when a region at the same height isextracted as a rectangular ring shape, the parts box 30 is detected withthe region of the rectangular ring shape being taken as a peripheralwall, and a monitoring area is set to include this region. In thepresent embodiment, it is assumed that a placement surface of the shelf2 on which the parts box 30 is placed is located horizontally relativeto the imaging unit 13.

Specifically, when two parts boxes 30 e and 30 f are placed on the shelf2 as shown in FIG. 38A, rectangular ring shapes are extracted as shownin the distance image shown in FIG. 38B, in which the upper end face ofthe peripheral wall of each parts box 30 is located at the same height.

Since a monitoring area corresponding to the parts box 30 can be set onthe basis of a region at the same height in the distance image, there isno need of providing a device for setting a monitoring area andperforming presetting or complicated terminal operations. Accordingly,it is possible to reduce a motion of the worker M required to set amonitoring area.

Further, another ToF camera may be provided separately from the imagingunit 13 to extract a region located at the same height. Further, when aplacement surface of the shelf 2 on which each parts box 30 is locatedis inclined relative to the imaging unit 13, the inclination angle canbe measured in advance so that the height can be corrected taking theinclination angle into consideration in the distance image. Further, thecharacteristic configuration of the present embodiment for setting amonitoring area corresponding to the parts box 30 on the basis of theregion located at the same height in the distance image can also beapplied to other embodiments and the like.

Twelfth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a twelfth embodiment will now bedescribed.

The twelfth embodiment differs from the first embodiment in the processfor setting the monitoring area. The components which are substantiallythe same as those of the first embodiment are denoted by the samereference signs, and the description thereof will be omitted.

In the present embodiment, the color of each parts box 30 has a specificcharacteristic so that the area occupied by each parts box 30 can beeasily extracted as a monitoring area from the captured image. The colorthat may be selected for each parts box 30 is unique to each parts box30 and easily distinguishable from the surrounding color such as theshelf 2 or the color of the hand of the worker M.

For example, as shown in FIG. 39A, in a captured image of the partsboxes 30 e and 30 f, the parts box 30 e may be a first color (forexample, green), and the parts box 30 f may be a second color (forexample, blue). When the monitoring area for the parts box 30 f is setby using this captured image, an area of the second color is extractedfrom the image as shown in FIG. 39B. Then, filtering such as expansionand contraction, including noise removal, is performed to the image inwhich the area of the second color is extracted. Thus, a rectangularring area is extracted as shown in FIG. 39C. Accordingly, an areaincluding the extracted rectangular ring area can be set as the abovemonitoring area. In FIGS. 39A to 39C, only the second color is hatched.

In particular, distinguishing the color for each parts box 30 canfacilitate recognition of the type of the parts box 30 corresponding tothe extracted monitoring area. Thus, by associating the color with thetype of the parts box 30, it is possible to detect what parts boxes 30are arranged in what order. Even if some parts 20 have similar colors,they can be distinguished by comparison with the right arrangement orderof the parts boxes 30 or the like. Depending on the environment in whichthe work analysis device 10 is installed, there may be an influence froman external lighting or the like. Therefore, a color space such as HSV,which has high brightness variations, can also be used to preventerroneous recognition.

Thirteenth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a thirteenth embodiment will now bedescribed.

The thirteenth embodiment differs from the first embodiment in theprocess for setting the monitoring area. The components which aresubstantially the same as those of the first embodiment are denoted bythe same reference signs, and the description thereof will be omitted.

In the present embodiment, prior to the work analysis processing,monitoring area setting is performed by the control unit 11 to set eachmonitoring area. In this monitoring area setting, an area in which anobject assumed as the part 20 is imaged in the image captured by theimaging unit 13 is defined as a parts area Pn. Then, an area of theparts box 30, that is, a monitoring area, is set on the basis of theparts area Pn.

Specifically, a single part image of the target part 20 or two or morepart images of the target part 20 with different view angles (template)is prepared in advance. As shown in FIG. 40, in an extraction image inwhich an area where the parts box 30 may be placed is extracted from theimage captured by the imaging unit 13, an area having high degree ofsimilarity to the above part image is searched (template matching). Inthe present embodiment, a detection window Sw is set according to thesize of the part 20 to be imaged, and an area having high degree ofsimilarity is searched by sliding the detection window Sw in the aboveextraction image.

Referring to the drawings, the monitoring area setting performed by thecontrol unit 11 in the present embodiment will now be specificallydescribed.

Prior to the work analysis processing, the monitoring area setting isinitiated by the control unit 11 in response to a predeterminedoperation while the parts box 30 is placed on the shelf 2. In initialsetting at step S401 in FIG. 41, the type of the parts box 30 for whicha monitoring area is to be set is specified, and the part image of thepart 20 contained in this parts box 30 is obtained. The part image maybe obtained from the storage unit 12, in which the part image ispre-stored, or may be externally obtained each time the monitoring areasetting is performed.

When all the monitoring areas are not set (No at S403), the imaging unit13 is ready to perform imaging (S405), and then parts area setting isperformed at step S407. In the subroutine of step S407, as shown in FIG.42, similar area search is performed at step S501. At step S501, asdescribed above, the detection window Sw is slid at a predeterminedinterval (for example, by each pixel or by several pixels) in the aboveextraction image to search an area having high degree of similarity (seeFIG. 40).

When search of an area having high degree of similarity is completed forall the extraction images, determination is performed at step S503 todetermine whether or not an area having degree of similarity equal to orhigher than a determination threshold has been included. In the presentembodiment, for example, the determination threshold is set to 0.8. Whenthe search result includes at least one area whose degree of similarityis equal to or higher than the determination threshold, it is determinedas “Yes” at step S503. Further, even if the search result does notinclude at least one area whose degree of similarity is equal to orhigher than the determination threshold (No at S503), it is determinedas “Yes” at step S505 as long as a predetermined number of (for example,10) areas whose degree of similarity is equal to or higher than apartial threshold (for example, 0.2) are included.

As described above, if determination at step S503 is “Yes” ordetermination at step S505 is “Yes,” setting is performed at step S507to set the parts area Pn on the basis of the searched area.Specifically, as shown in FIG. 43, a rectangular area including all theareas whose degree of similarity is equal to or higher than the abovepartial threshold is set as the parts area Pn, and the subroutine ofparts area setting ends. In FIG. 43, some of the areas whose degree ofsimilarity is equal to or higher than the above partial threshold areillustrated as the rectangles indicated by the dotted line.

On the other hand, if the search result does not include an area whosedegree of similarity is equal to or higher than the determinationthreshold (No at S503 in FIG. 42) and does not include the predeterminednumber of areas whose degree of similarity is equal to or higher thanthe partial threshold (No at S505), the subroutine of parts area settingends without setting the parts area Pn.

As described above, when the subroutine of parts area setting shown inFIG. 42 ends, determination is performed at step S409 in FIG. 41 todetermine whether or not the parts area Pn has been set. If the partsarea Pn has not been set, it is determined as “No” at step S409, andnotification is performed at step S411. At step S411, notification isperformed to notify by the displaying unit 14 that the part 20 of thesearch target is not found in the captured image, and the monitoringarea setting ends.

On the other hand, if the parts area Pn has been set (Yes at S409),straight line search is performed at step S413. At step S413, straightline is searched toward outside from a point on the edge of the partsarea Pn in the extraction image. In the present embodiment, as shown inFIG. 43, straight line is searched from the minimum X coordinate and Ycoordinate in the parts area Pn in the negative X direction (seereference numeral L1 in FIG. 43). Since the parts area Pn is set withinthe area corresponding to the parts box 30, a straight linecorresponding to the inner surface of the peripheral wall of the partsbox 30 is searched by this straight line search. In addition, it is notlimited to searching from the minimum X coordinate and Y coordinate inthe parts area Pn in the negative X direction, and a straight line canalso be searched toward outside from any point on the edge of the partsarea Pn.

When the straight line is searched as described above, intersectionsearch is performed at step S415 to search an intersection where a tipof the straight line that has been searched intersects with anotherstraight line in the extraction image. When an intersection is searchedclockwise or counterclockwise on the straight line, the number ofsearched intersections, or k, is initialized to 0 (S417). Then, if theintersection is outside the parts area Pn (Yes at S419), the number ofsearched intersections k is incremented (S421). If it is immediatelyafter the number of intersections k is set to 0, the number ofintersections k is set to 1.

If the number of intersections k is not 4 (No at S423), nextintersection, which is the tip of another straight line continuous fromthe intersection is searched (S425). If the searched intersection isoutside the parts area Pn (Yes at S419), and the number of incrementedintersections k is not 4 (No at S423), the steps from step S425 onwardare repeated. Further, if the searched intersection is inside the partsarea Pn (No at S419), it is determined that an erroneous intersectionhas been searched. Then, search for an intersection on the straight lineis continuously performed (S427).

Then, next intersection is searched (S425), and the number ofincremented intersections k becomes 4 (Yes at S423), setting isperformed at step S429. At step S429, since the rectangular area havingfour corners at four searched intersections corresponds to a rectangulararea surrounded by the peripheral wall of the parts box 30, setting isperformed to set the above rectangular area as a monitoring area. Then,after completion of step S429, the steps from step S403 onward areperformed to set a monitoring area of the next parts box 30. When amonitoring area for all the parts boxes 30 are set, it is determined as“Yes” at step S403, and the monitoring area setting ends.

In the monitoring area setting of the present embodiment, an area inwhich an object assumed as the part 20 is imaged is defined as a partsarea Pn. Then, an area of the parts box 30, that is, a monitoring area,is set on the basis of the parts area Pn. Accordingly, there is no needof providing a device for setting a monitoring area and performingpresetting or complicated terminal operations, so it is possible toreduce a motion of the worker M required to set a monitoring area.

The parts area Pn is not limited to being set by using the degree ofsimilarity to the part image, and other technique may also be used. Forexample, in a first modified example of the present embodiment, theparts area Pn can be set on the basis of the detection result obtainedby using an edge detection algorithm (for example, Canny method).

Specifically, luminance is line-scanned in the X coordinate directionfor the gray scale in the above extraction image (see FIG. 44). The Ycoordinates of the start and end of a section in which an amplitude iscontinuously detected for a threshold number of times (e.g., four times)or more within a predetermined section (e.g., within 10 pixels) isstored. Further, luminance is also line-scanned in the Y coordinatedirection (see FIG. 44), the X coordinates of the start and end of thedetected section is stored. FIG. 45 conceptually illustrates a detectionresult obtained by line-scanning the luminance in the X coordinatedirection with respect to a predetermined Y coordinate. Among the storedX coordinates and the Y coordinates, the coordinate closest to theorigin is set as a point through which two of the four straight linesforming the outer edge of the parts area Pn pass, and the coordinatefarthest from the origin is set as a point through which the remainingtwo straight lines pass. Thus, the parts area Pn is set. The aboveeffect is also achieved even when a monitoring area is set on the basisof the parts area Pn that has been set in the above described manner.

Further, in a second modified example of the present embodiment, theparts area Pn can be set on the basis of the detection result obtainedby using a corner detection algorithm (for example, Harris Cornerdetection).

Specifically, in the above extraction image, the corners of each objectare detected, and an angle of a smaller corner (less than 180 degrees)formed by two straight lines intersecting each other at the detectedcorners and the coordinates thereof are detected (see the hatched areain FIG. 46). Then, the histogram is calculated for each approximatecorner angle (see FIG. 47). The parts area Pn is set so as to correspondto an area surrounded by the X coordinates and Y coordinates closest tothe origin and the X coordinates and Y coordinates farthest from theorigin in the angles whose frequency of appearance in the histogram iswithin a predetermined number from the top (the number depends on theshape of the part, and, for example, top three for a triangular shape).The above effect is also achieved even when a monitoring area is set onthe basis of the parts area Pn that has been set in the above describedmanner.

The characteristic configuration of the present embodiment for settingthe parts area Pn to set a monitoring area by using the parts area Pncan also be applied to other embodiments and the like.

The present invention is not limited to the first to thirteenthembodiments and modified examples thereof, and, for example, may beembodied as described below.

(1) The predetermined work to be analyzed is not limited to assemblywork composed of four fundamental work operations A to D. For example,assembly work composed of one to three fundamental work operations maybe adopted, or assembly work composed of five or more fundamental workoperations may be adopted.

(2) The aforementioned delimitation motion (a predetermined motion) isnot limited to a motion of taking out the part 20 from the parts box 30(first embodiment), or a motion of transferring the part 20 to anassembly position in the workpiece W (second embodiment). Thedelimitation motion may also be another motion that is essential for thefundamental work operation, or a motion whose image is easily recognizedand that is irrelevant to the fundamental work operation.

(3) The invention can be applied not only to work analysis in assemblyof the parts to the workpiece W such as a printed circuit boardaccording to a predetermined work procedure, but also to work analysisin assembly of the parts to a semi-finished product, for example, in anassembly line for high-mix low-volume products according to apredetermined work procedure.

(4) Information or delimitation information of the fundamental workoperation determined to be abnormal in the work analysis processing canbe displayed on the displaying unit 14 or the like to provide anopportunity for correcting the delimitation information, if necessary,by a manual operation.

Fourteenth Embodiment

With reference to the drawings, a work analysis device and a workanalysis program according to a fourteenth embodiment will now bedescribed.

In the present embodiment, whether or not the fundamental work operationis performed is determined on the basis of the reliability set for eachmonitoring area. The components which are substantially the same asthose of the first embodiment are denoted by the same reference signs,and the description thereof will be omitted.

Similarly to the first embodiment, the work analysis device 10 accordingto the present embodiment is installed on a place such as the work table1 shown in FIG. 1, and is configured as a device that acquires a workvideo image for work analysis by repeatedly capturing predeterminedwork, in which a plurality of fundamental work operations are performedin a predetermined order by the worker M.

In the present embodiment, as shown in FIG. 48, markers 31 a to 31 d areprovided at four corners on the upper end face of the parts boxes 30 ato 30 d, respectively, so that the ranges of accommodating the parts inthe parts boxes are recognized from the captured image. The markers 31 ato 31 d are different for each terminal box. In the captured image, arange having the markers 31 a as four corners is set as a monitoringarea P11 a, a range having the markers 31 b as four corners is set as amonitoring area P11 b, a range having the markers 31 c as four cornersis set as a monitoring area P11 c, and a range having the markers 31 das four corners is set as a monitoring area P11 d. In the presentembodiment, the marker 31 a is formed as a black circle, the marker 31 bis formed as a black star shape, the marker 31 c is formed as a blackdiamond shape, and the marker 31 d is formed as an asterisk shape.However, these shapes are merely examples, and any shape can also beused as long as four corners of the parts box can be recognized from thecaptured image. Further, FIG. 48 is an enlarged view of a portion nearthe parts boxes 30 a to 30 d in an image captured by the imaging unit13, which will be described later.

The following description will be given of the work analysis processingperformed on the basis of the work analysis program, which is performedby the control unit 11, when the worker M performs the predeterminedwork by which parts contained in a plurality of parts boxes areassembled to the workpiece W in sequence according to a predeterminedwork procedure.

In the present embodiment, the predetermined work to be analyzedincludes fundamental work operations E to H, by which four types ofparts 20 a to 20 d are assembled in sequence to the workpiece W in thestate shown in FIG. 49A, respectively, to thereby provide the workpieceW in the state shown in FIG. 49B. The fundamental work operation E istaking out the part 20 a from the parts box 30 a and assembling the part20 a that has been taken out to an assembly position Wa in the workpieceW. The fundamental work operation F is taking out the part 20 b from theparts box 30 b and assembling the part 20 b that has been taken out toan assembly position Wb in the workpiece W. The fundamental workoperation G is taking out the part 20 c from the parts box 30 c andassembling the part 20 c that has been taken out to an assembly positionWe in the workpiece W. The fundamental work operation H is taking outthe part 20 d from the parts box 30 d and assembling the part 20 d thathas been taken out to an assembly position Wd in the workpiece W. In thepresent embodiment, according to a predetermined work procedure, thefundamental work operation E, the fundamental work operation F, thefundamental work operation G, and the fundamental work operation H areperformed in this order.

As shown in FIGS. 48 and 49, a first monitoring area for detecting astart motion of the fundamental work operation E (a motion of taking outthe part 20 a from the parts box 30 a) is set as a monitoring area P11a, which is a range having the markers 31 a as four corners thereof (arange corresponding to the parts box 30 a) in the imaging range of theimaging unit 13. Further, a second monitoring area for detecting acompletion motion of the fundamental work operation E (a motion ofassembling the part 20 a to the assembly position Wa in the workpiece W)is set as a monitoring area P21 a, which is a range around the assemblyposition Wa in the imaging range of the imaging unit 13.

In addition, a first monitoring area for detecting a start motion of thefundamental work operation F (a motion of taking out the part 20 b fromthe parts box 30 b) is set as a monitoring area P11 b, which is a rangehaving the markers 31 b as four corners thereof (a range correspondingto the parts box 30 b) in the imaging range of the imaging unit 13.Further, a second monitoring area for detecting a completion motion ofthe fundamental work operation F (a motion of assembling the part 20 bto the assembly position Wb in the workpiece W) is set as a monitoringarea P21 b, which is a range around the assembly position Wb in theimaging range of the imaging unit 13. In addition, a first monitoringarea for detecting a start motion of the fundamental work operation G (amotion of taking out the part 20 c from the parts box 30 c) is set as amonitoring area P11 c, which is a range having the markers 31 c as fourcorners thereof (a range corresponding to the parts box 30 c) in theimaging range of the imaging unit 13.

Further, a second monitoring area for detecting a completion motion ofthe fundamental work operation G (a motion of assembling the part 20 cto the assembly position Wc in the workpiece W) is set as a monitoringarea P21 c, which is a range around the assembly position Wc in theimaging range of the imaging unit 13. In addition, a first monitoringarea for detecting a start motion of the fundamental work operation H (amotion of taking out the part 20 d from the parts box 30 d) is set as amonitoring area P11 d, which is a range having the markers 31 d as fourcorners thereof (a range corresponding to the parts box 30 d) in theimaging range of the imaging unit 13. Further, a second monitoring areafor detecting a completion motion of the fundamental work operation H (amotion of assembling the part 20 d to the assembly position Wd in theworkpiece W) is set as a monitoring area P21 d, which is a range aroundthe assembly position Wd in the imaging range of the imaging unit 13.Each start motion described above corresponds to an example of a “firstmotion,” and each completion motion described above corresponds to anexample of a “second motion.”

In the work analysis processing in which the monitoring areas P11 a toP11 d and P21 a to P21 d are set as described above, a reliability(hereinafter, also referred to as a first reliability) is set in amanner to become higher with an increase in probability of determiningthat a motion of taking out the part from the monitoring area (partsbox) is performed, on the basis of comparison result of the imagescaptured before and after the hand of the worker M enters the monitoringareas P11 a to P11 d, which are set as the first monitoring areas.Further, a reliability (hereinafter, also referred to as a secondreliability) is set in a manner to become higher with an increase inprobability of determining that a motion of assembling the part to themonitoring area (assembly position of the workpiece W) is performed, onthe basis of comparison result of the images captured before and afterthe hand of the worker M enters the monitoring areas P21 a to P21 d,which are set as the second monitoring areas.

Then, the reliability set as described above are each quantified, and,on the basis of the sum of two reliability values, it is determinedwhether or not the corresponding fundamental work operations areperformed. When the determination is positive, the delimitationinformation for delimiting the work video image is set so that thetiming of having captured the image for setting the reliability in thefirst monitoring area is set as a work start timing, and the timing ofhaving captured the image for setting the reliability in the secondmonitoring area is set as a work completion timing.

The monitoring areas P11 a to P11 d is not limited to being set by usingthe markers 31 a to 31 d provided at four corners on the upper end faceof the parts boxes 30 a to 30 d as described above each time the workanalysis processing is performed, but may also be set on the basis ofthe image difference generated by continuously capturing the parts boxes30 a to 30 d while they are individually swinging. The monitoring areasP11 a to P11 d can be set as standard ranges without requiring an imageanalysis or the like, under the assumption that the parts boxes 30 a to30 d, each having a predetermined shape, are placed in position.

Referring to a flowchart of FIGS. 50 to 52, the work analysis processingperformed by the control unit 11 will now be described in detail.

The control unit 11 starts the work analysis processing when apredetermined start operation is performed to the operation unit 17. Atstep S601 in FIG. 50, work information reading is performed to read thework information on the specified predetermined work (information on thetypes and amount (predetermined amount) of the parts used for the work,the types of workpiece W, the monitoring area, and the like) from thestorage unit 12. The work information includes information on whetherthe amount of the parts taken out is countable on the basis of the imagedifference in the imaging unit 13 when the target parts are taken outfrom the parts box. That is, when the parts are large enough to countthe number of parts taken out on the basis of the image difference,countable information is imparted, whereas, when the parts are too smallto count the number of parts taken out on the basis of the imagedifference, uncountable information is imparted. Further, in the presentembodiment, the above work information is stored in the storage unit 12each time an information code or an RF tag is read from the workinstruction or the like, in which the work procedure for themanufacturing lot is recorded. Further, the above work information canbe stored in the storage unit 12 via an input operation or the like fromthe operation unit 17, or may be stored in the storage unit 12 when itreceives the work information from a higher level device or the like viathe communication unit 18.

When the work information is read as described above, monitoring areasetting is performed at step S603. At step S603, in a captured image ina state in which imaging of the work performed by the worker M is imagedas a work video image by the imaging unit 13, setting of the firstmonitoring area and the second monitoring area is performed for eachfundamental work operation. First, for the fundamental work operation E,which is the first fundamental work operation, a range having themarkers 31 a, which are specified in the image captured by the imagingunit 13, as four corners is set as a monitoring area (first monitoringarea) P11 a, and a range around the assembly position Wa in the imagingrange of the imaging unit 13 is set as a monitoring area (secondmonitoring area) P21 a. Further, the control unit 11 that performsmonitoring area setting at step S603 can correspond to an example of a“monitoring area setting unit.”

When the first monitoring area and the second monitoring area are set asdescribed above, first reliability setting is performed at step S605. Atstep S605, first reliability setting is performed to set the firstreliability on the basis of the comparison result of the images capturedbefore and after the hand of the worker M enters the monitoring area P11a.

Specifically, in determination at step S701 in FIG. 51, it is determinedwhether or not the hand of the worker M has been present in themonitoring area P11 a for a predetermined period of time, by using avideo image acquired by the imaging unit 13. It is determined as “No”repeatedly until an image difference occurs in the monitoring area P11a. If it is determined that the hand of the worker M has been present inthe monitoring area P11 a for a predetermined period of time accordingto the image difference (Yes at S701), it is determined whether or notthe parts to be taken out are countable in determination at step S703.That is, in the present embodiment, setting method for the firstreliability in the first monitoring area is changed depending on whetherthe parts are countable or not. If the work information read asdescribed above includes the countable information, which is imparted tothe parts to be taken out from the monitoring area P11 a, it isdetermined that the parts are countable and determination at step S703is “Yes.”

Subsequently, in determination at step S705, it is determined whetherthe standard number of parts has decreased or not. On the basis of theimage difference, which is a comparison result between the imagecaptured immediately before the hand enters the monitoring area P11 aand the image captured immediately after the hand exits the monitoringarea P11 a, if the number of parts counted is attributed to a decreaseof the standard number of parts, which is recognized from the above workinformation, the probability that the standard number of parts 20 a havebeen taken out from the monitoring area P11 a is determined to be high,and determination at step S705 is “Yes.” In this case, the firstreliability is set to “High,” which is highest as the settablereliability (S707).

Further, on the basis of the image difference between the image capturedimmediately before the hand enters the monitoring area P11 a and theimage captured immediately after the hand exits the monitoring area P11a, if the number of parts counted is not attributed to a decrease of thestandard number of parts, for example, if the number of parts counted isunchanged, the probability that the standard number of parts 20 a havebeen taken out from the monitoring area P11 a is determined to be low,and determination at step S705 is “No.” In this case, the firstreliability is set to “Lo,” which is lowest as the settable reliability(S709).

On the other hand, if the work information read as described aboveincludes the uncountable information, which is imparted to the parts tobe taken out from the monitoring area P11 a, determination at step S703is “No.” In this case, the first reliability is set to “Mid,” which islower than “High” and higher than “Lo” in the settable reliability(S711).

Among the images used for the above image difference, the timing (time)of having captured the image immediately before the hand enters can beset as a work start timing. When the first reliability is set to any oneof the “High,” “Mid,” and “Lo,” the first reliability is registeredusing the database of the storage unit 12 (S713) together with the abovework start timing, and the first reliability setting ends. Further,among the images used for the above image difference, the timing (time)of having captured the image immediately after the hand exits may alsobe set as a work start timing depending on the work content or the like.

Subsequently, second reliability setting is performed at step S607 inFIG. 50. At step S607, second reliability setting is performed to setthe second reliability on the basis of the comparison result of theimages captured before and after the hand of the worker M enters themonitoring area P21 a.

Specifically, in determination at step S801 in FIG. 52, it is determinedwhether or not the hand of the worker M has been present in themonitoring area P21 a for a predetermined period of time, by using avideo image acquired by the imaging unit 13. It is determined as “No”repeatedly until an image difference occurs in the monitoring area P21a. If it is determined that the hand of the worker M has been present inthe monitoring area P21 a for a predetermined period of time accordingto the image difference (Yes at S801), it is determined whether or notthe part 20 a has been assembled to the assembly position Wa in theworkpiece W in determination at step S803.

On the basis of the comparison result (image difference) between theimage captured immediately before the hand enters the monitoring areaP21 a and the image captured immediately after the hand exits themonitoring area P21 a, when the image difference substantially matchesthe image of the part 20 a assembled to the assembly position Wa, theprobability that the part 20 a is assembled to the assembly position Waof the workpiece W is determined to be high. Then, it is determined as“Yes” at step S803. When the determination at step S803 is “Yes,” thesecond reliability is set to “High,” which is highest as the settablereliability (S807).

On the basis of the comparison result (image difference) between theimage captured immediately before the hand enters the monitoring areaP21 a and the image captured immediately after the hand exits themonitoring area P21 a, when the image difference obviously does notmatch the image of the part 20 a assembled to the assembly position Wa,the probability that the part 20 a is assembled to the assembly positionWa of the workpiece W is determined to be low. Then, it is determined as“No” at step S803, and “Yes” at step S805. When the determination atstep S803 is “No” and determination at step S805 is “Yes,” the secondreliability is set to “Lo,” which is lowest as the settable reliability(S809).

On the other hand, on the basis of the comparison result (imagedifference) between the image captured immediately before the handenters the monitoring area P21 a and the image captured immediatelyafter the hand exits the monitoring area P21 a, there may be a casewhere the probability that the part 20 a is assembled to the assemblyposition Wa of the workpiece W cannot be determined to be high or low.For example, there may be cases where the part 20 a is assembled in astate deviated from the assembly position Wa, a different part having acolor or a shape similar to that of the part 20 a is assembled to theassembly position Wa, or the part 20 a is so small that the image of thepart 20 a cannot be clearly recognized. In such a case, determination atstep S803 is No and determination at step S805 is No, and the secondreliability is set to “Mid” (S811).

Among the images used for the above image difference, the timing (time)of having captured the image immediately after the hand exits can be setas a work completion timing. When the second reliability is set to anyone of the “High,” “Mid,” and “Lo,” the second reliability is registeredusing the database of the storage unit 12 (S813) together with the abovework completion timing, and the second reliability setting ends.Further, among the images used for the above image difference, thetiming (time) of having captured the image immediately before the handenters may also be set as a work completion timing depending on the workcontent or the like.

In a modified example of the second reliability setting, for example,for countable parts, it is determined whether or not the imagedifference substantially matches the image of the part assembled to theassembly position. When the determination is positive, the secondreliability may be set to “High,” and, when the determination isnegative, the second reliability may be set to “Lo.” For uncountableparts, the second reliability may be set to “Mid.” In addition, thecontrol unit 11 that performs the first reliability setting at step S605and the second reliability setting at step S607 corresponds to anexample of a “reliability setting unit.”

When the first reliability and the second reliability are set asdescribed above, total reliability setting is performed at step S609. Inthis step, a sum of a reliability score obtained by quantifying thefirst reliability and a reliability score obtained by quantifying thesecond reliability is set as a total reliability. Specifically, forexample, as shown in FIG. 53, “High,” “Mid,” and “Lo” in the firstreliability are quantified as “2,” “1,” and “0,” respectively, and“High,” “Mid,” and “Lo” in the second reliability are quantified as “3,”“2,” and “1,” respectively, so as to be weighted more than the firstreliability. Then, these scores are summed (added) together to set(calculate) a total reliability. For example, as shown in FIG. 53, whenthe first reliability and the second reliability are both set to “High,”the total reliability is set to “5.”

Subsequently, in determination at step S611 in FIG. 50, it is determinedwhether the reliability is equal to or higher than the standard value.In the present embodiment, the standard value is set to “2” (see FIG.53). For example, when both the first reliability and the secondreliability are “High” and thus the total reliability becomes equal toor higher than the standard value (Yes at S611), it is determined thatthe fundamental work operation E is performed. Then, delimitationinformation setting is performed at step S613. At step S613, for thefundamental work operation E, which is the monitoring target, thedelimitation information for delimiting the work video image is set sothat an imaging timing associated with the first reliability is set as awork start timing, and an imaging timing associated with the secondreliability is set as a work completion timing, and the delimitationinformation is stored in the storage unit 12. Further, the control unit11 that performs determination at step S611 corresponds to an example ofa “determination unit,” and the control unit 11 that performsdelimitation information setting at step S613 corresponds to an exampleof a “delimitation information setting unit.”

When the delimitation information is set as described above, and thework is not completed (No at S617), the next fundamental work operation,or the fundamental work operation F, is set as the monitoring target(S619). Then, the steps from step S603 onward are performed to thefundamental work operation F. In this case, step S603 is performed tothe fundamental work operation F. At step S603, a range having themarkers 31 b, which are specified in the image captured by the imagingunit 13, as four corners is set as a monitoring area (first monitoringarea) P11 b, and a range around the assembly position Wb in the imagingrange of the imaging unit 13 is set as a monitoring area (secondmonitoring area) P21 b. Then, at step S605, first reliability setting isperformed to set the first reliability on the basis of the comparisonresult of the images captured before and after the hand of the worker Menters the monitoring area P11 b. Subsequently, at step S607, secondreliability setting is performed to set the second reliability on thebasis of the comparison result of the images captured before and afterthe hand of the worker M enters the monitoring area P21 b.

Then, a sum of a reliability score obtained by quantifying the firstreliability and a reliability score obtained by quantifying the secondreliability is set as a total reliability (S609). When the totalreliability thus determined becomes equal to or higher than the standardvalue (Yes at S611), it is determined that the fundamental workoperation F is performed. Then, for the fundamental work operation F,the delimitation information for delimiting the work video image is setso that an imaging timing associated with the first reliability is setas a work start timing, and an imaging timing associated with the secondreliability is set as a work completion timing, and the delimitationinformation is stored in the storage unit 12 (S613).

Subsequently, for the fundamental work operation G, the firstreliability is set in the same manner as above on the basis of thecomparison result of the images captured before and after the hand ofthe worker M enters the monitoring area P11 c (S605), and the secondreliability is set on the basis of the comparison result of the imagescaptured before and after the hand of the worker M enters the monitoringarea P21 c (S607). Thus, the total reliability is set (S609). When thetotal reliability thus determined becomes equal to or higher than thestandard value (Yes at S611), it is determined that the fundamental workoperation G is performed. Then, for the fundamental work operation G,the delimitation information for delimiting the work video image is setso that an imaging timing associated with the first reliability is setas a work start timing, and an imaging timing associated with the secondreliability is set as a work completion timing, and the delimitationinformation is stored in the storage unit 12 (S613).

Similarly, for the fundamental work operation H, the first reliabilityis set in the same manner as above on the basis of the comparison resultof the images captured before and after the hand of the worker M entersthe monitoring area P11 d (S605), and the second reliability is set onthe basis of the comparison result of the images captured before andafter the hand of the worker M enters the monitoring area P21 d (S607).Thus, the total reliability is set (S609). When the total reliabilitythus determined becomes equal to or higher than the standard value (Yesat S611), it is determined that the fundamental work operation H isperformed. Then, for the fundamental work operation H, the delimitationinformation for delimiting the work video image is set so that animaging timing associated with the first reliability is set as a workstart timing, and an imaging timing associated with the secondreliability is set as a work completion timing, and the delimitationinformation is stored in the storage unit 12 (S613).

As described above, even after the delimitation information is set asthe predetermined work for four fundamental work operations E to H, thefundamental work operations E to H are repeatedly performed.Accordingly, the delimitation information is set for each fundamentalwork operation. Since the delimitation information is set for a numberof fundamental work operations, it is possible to generate data forsupporting the work such as work procedure check and cycle timemeasurement by simply analyzing the work video image delimited by usingthe delimitation information. Further, the determination data(annotation data) for determining whether or not the correct work isperformed can be automatically generated.

On the other hand, when the total reliability becomes less than thestandard value (No at S611), temporary delimitation information settingis performed at step S615. At step S615, unlike the delimitationinformation setting at step S613, the temporary delimitation informationis set. That is, for the fundamental work operation with the totalreliability of less than the standard value, a temporary delimitationinformation is separately set from the delimitation information so thatan imaging timing associated with the first reliability is set as atemporary work start timing, and an imaging timing associated with thesecond reliability is set as a temporary work completion timing, and thetemporary delimitation information is stored in the storage unit 12.

The temporary delimitation information stored in the storage unit 12together with the reliability can be displayed on the displaying unit 14or the like in an editable manner after completion of the work so thatthe worker or the like can edit the temporary delimitation informationwhile watching the work video image to thereby obtain correctdelimitation information. In particular, since the temporarydelimitation position is graphically displayed, the worker or the likecan check the work delimitation having low reliability, and the workdelimitation that has failed in correct determination.

As described above, according to the work analysis device 10 of thepresent embodiment, the first monitoring area (P11 a to P11 d) fordetecting the start motion (first motion) of the fundamental workoperation and the second monitoring area (P21 a to P21 d) for detectingthe completion motion (second motion) of the fundamental work operationare set for each fundamental work operation in the imaging range of theimaging unit 13. On the basis of the comparison result, among the imagescaptured by the imaging unit 13, between a portion corresponding to themonitoring area and a portion corresponding to the monitoring area inanother image that is acquired before the above image, the reliability,which becomes higher when it is determined that the motion related tothe fundamental work operation is performed in the monitoring area, isset for each monitoring area. Then, on the basis of the firstreliability set at the first monitoring area and the second reliabilityset at the second monitoring area, it is determined whether thecorresponding fundamental work operation is performed or not. For thefundamental work operation that is determined that the work isperformed, the delimitation information for delimiting the work videoimage acquired by the imaging unit 13 for each fundamental workoperation is set so that the timing of having captured the image forsetting the reliability in the first monitoring area is set as a workstart timing, and the timing of having captured the image for settingthe reliability in the second monitoring area is set as a workcompletion timing.

Thus, whether the corresponding fundamental work operation is performedor not is determined on the basis of the first reliability set at thefirst monitoring area and the second reliability set at the secondmonitoring area. Accordingly, compared with the case where determinationis made on the basis of a monitor result at a single monitoring area, animproved determination accuracy can be obtained. In particular, thereliability scores at multiple positions are used as determinationcriteria. In this case, even if low reliability is set at one monitoringarea due to delicate fundamental work operation being performed, highreliability is set at another monitoring area, so it is possible todetermine that the fundamental work operation is being performed.Accordingly, a wide variety of fundamental work operations includingdelicate work can be determined with high accuracy. Therefore, the workanalysis device that can delimit the work video image with highaccuracy, even for the work that cannot be clearly determined whetherthe part has been taken out or not, can be achieved.

In particular, a fundamental work operation is assembling a parts 20 ato 20 d, which has been taken out of the predetermined parts boxes 30 ato 30 d, respectively, to the workpiece (assembly target) W, and thestart motion is a motion of taking out the part from the parts box, andthe completion motion is a motion of assembling the part to the assemblyposition of the assembly target. Accordingly, on the basis of thecomparison result of the captured images of the first monitoring area(P11 a to P11 d), the first reliability that is set at the firstmonitoring area increases with an increase in the probability ofdetermining that a motion of taking out the part from the parts box isperformed. Further, on the basis of the comparison result of thecaptured images of the second monitoring area (P21 a to P21 d), thesecond reliability that is set at the second monitoring area increaseswith an increase in the probability that a motion of assembling the partto the assembly position in the assembly target is performed.

As described above, since the reliability on the basis of the motion oftaking out the part from the parts box and the motion of assembling thepart to the assembly position of the assembly target are both taken intoconsideration, the work video image can be delimited with high accuracyeven when a fundamental work operation of taking out a part from apredetermined parts box and assembling the part to the assembly targetis repeatedly performed. This contributes to more efficient and moreaccurate analysis of the state of work manually performed by the worker,for example, part assembly work (manual work).

In the first reliability setting, setting method for the firstreliability related to the first monitoring area is changed depending onwhether the parts are countable or not on the basis of the imagedifference of the images of the first monitoring area captured by theimaging unit 13. That is, for the parts that are countable, thereliability is set to “High” when the standard number of parts hasdecreased, the reliability is set to “Lo” when the standard number ofparts has not decreased, and the reliability is set to “Mid” when theparts are uncountable. Thus, the reliability can be set moreappropriately by changing the reliability setting method suitable forthe parts to be monitored.

Further, in the work analysis processing, whether the correspondingfundamental work operation is performed or not is determined on thebasis of the total reliability (sum), which is obtained by quantifyingand summing up all the reliability while weighting the secondreliability set at the second monitoring area (P21 a to P21 d) more thanthe first reliability set at the first monitoring area (P11 a to P11 d).Accordingly, in determination using the total reliability, since aninfluence of the second reliability related to the motion of assemblingthe parts 20 a to 20 d to the assembly positions Wa to Wd in theworkpiece W becomes higher than an influence of the first reliabilityrelated to the motion of taking out the parts 20 a to 20 d from theparts boxes 30 a to 30 d, it is possible to set the reliability moresuitable for the actual operation.

The fourteenth embodiment is not limited to the one described above, andmay also be embodied as described below.

(1) The work analysis processing may be performed to the work videoimage acquired by the imaging unit 13 in real time, or may be performedafter the predetermined work ends. In the work analysis processingperformed in real time, when low reliability is set, predeterminednotification can be performed by using the light emitting unit 15, thespeaker 16, or the like to notify the worker or the like of theprobability of erroneous work.

(2) In the work analysis processing, the total reliability is notlimited to being set on the basis of the reliability scores set for twomonitoring areas (the first monitoring area and the second monitoringarea). Three or more monitoring areas including the first monitoringarea and the second monitoring area may also be set, and the totalreliability may be set according to the reliability that is set for eachmonitoring area.

(3) The reliability score used for the total reliability setting may beset as any numerical value depending on the content of work or the like,on the assumption that the numerical value decreases in the order of“High,” “Mid,” and “Lo.” Further, the reliability scores correspondingto “High,” “Mid,” and “Lo” may be set as the same numerical value in thefirst reliability and the second reliability in order to eliminate theabove weight depending on the content of work or the like. Thereliability score used for the total reliability setting is not limitedto three levels of “High,” “Mid,” and “Lo,” and may also be four or morelevels. Specifically, for example, four levels of “High,” “Mid1,”“Mid2,” and “Lo” may also be used. Alternatively, the reliability for aplurality of monitoring areas may be set by obtaining each reliabilityas a numerical value, and by multiplying the numerical value by acoefficient or the like.

(4) In addition, in the embodiments, the number of fundamental workoperations is not limited to four, but may be set to two, three, or fiveor more fundamental work operations. As a modification, a condition ofusing only one fundamental work operation can be provided.

(5) The fundamental work operation as the monitoring target is notlimited to the work by which the part that has been taken out from theparts box is assembled to the workpiece W such as a printed circuitboard, and may also be, for example, the work by which the part isassembled to a semi-finished product in an assembly line for high-mixlow-volume products. Further, the fundamental work operation as themonitoring target may also be, for example, the work by which the partis deformed or removed from the assembly target.

(6) The imaging timing for setting the first reliability and the imagingtiming for setting the second reliability are not limited to being setas a work start timing and a work completion timing, respectively. Oneof these timings may be set as a work completion timing of the precedingfundamental work operation (for example, fundamental work operation E)and a work start timing of the subsequent fundamental work operation(for example, fundamental work operation F).

Fifteenth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to a fifteenth embodiment will now be described.

The fifteenth embodiment mainly differs from the first embodiment in theprocess for setting the monitoring area. The components which aresubstantially the same as those of the first embodiment are denoted bythe same reference signs, and the description thereof will be omitted.

As illustrated in FIG. 54, a work support device 100 according to thepresent embodiment is installed on the work table 1 or the like, and isconfigured as a device that supports assembly work performed by theworker M by using parts contained in a plurality of parts boxesaccording to a predetermined work procedure.

As shown in FIGS. 54 and 55, the work support device 100 includes thecontrol unit 11, the storage unit 12, the imaging unit 13, thedisplaying unit 14, the light emitting unit 15, the speaker 16, theoperation unit 17, the communication unit 18, and the like. The controlunit 11 (CPU 11A) is mainly composed of a microcomputer, and performsoverall control of the work support device 100 and various calculationsas well as work support processing as described later. Since thesecomponents have the same configuration as those described above inconnection with FIG. 3, the following description will be given,focusing on the main differences.

The imaging unit 13 is configured as a camera having a light receivingsensor (for example, C-MOS area sensor or CCD area sensor) as describedabove. In the present embodiment, the imaging unit 13 is separatelyprovided from a device main body 100 a which includes the control unit11, the displaying unit 14, and the like, and is disposed in an upperpart of the work table 1 so as to acquire a video image or a still imageof a range at least including the parts boxes 30 a to 30 d. In thepresent embodiment, the imaging unit 13 is configured to acquire asequence of still images, for example, at 30 frame per second, and storethe acquired images in the storage unit 12 so as to be analyzed by thecontrol unit 11.

As described above, the displaying unit 14 is a liquid crystal display,for example, and is controlled by the control unit 11 to display animage acquired by the imaging unit 13, predetermined information, andthe like. The device main body 100 a is mounted on a rear plate or thelike of the work table 1 so that the worker M can see the display screenof the displaying unit 14.

The light emitting unit 15 and the speaker 16 having the sameconfiguration as in the first embodiment correspond to an example of a“notification unit.”

Referring to the flowchart in FIGS. 56 and 57, the following detaileddescription will be given of the work support processing performed onthe basis of the work support program, which is performed for eachmanufacturing lot by the control unit 11 (CPU 11A), when the worker Mperforms predetermined work by which parts contained in a plurality ofparts boxes are individually assembled to the workpiece W according to apredetermined work procedure. In the example of the predetermined workprocedure described below, the work procedure is fixed in advance, bywhich the part 20 a in the parts box 30 a, the part 20 b in the partsbox 30 b, the part 20 c in the parts box 30 c, and the part 20 d in theparts box 30 d are sequentially assembled to the workpiece W in thisorder.

The control unit 11 starts the work support processing when apredetermined start operation is performed to the operation unit 17. Atstep S901 in FIG. 56, work procedure information reading is performed toread information on the predetermined work procedure (hereinafter, alsoreferred to as “work procedure information”) of the specifiedmanufacturing lot from the storage unit 12. In the present embodiment,the above work procedure information is stored in the storage unit 12each time an information code or an RF tag is read from the workinstruction or the like, in which the work procedure for the specifiedmanufacturing lot is recorded. Further, the above work procedureinformation can be stored in the storage unit 12 via an input operationor the like from the operation unit 17, or may be stored in the storageunit 12 when it receives the work procedure information from a higherlevel device or the like via the communication unit 18.

When the work procedure information is read as described above, k is setto 1, where k is the assembly order (order of taking out) of the parts(S903), and then monitoring area setting is performed at step S905. Atstep S905, among the images acquired by the imaging unit 13, an imagearea corresponding to the parts box of the kth part in assembly order,which is specified from the work procedure information, is set as amonitoring area. When the assembly order k is set to 1 as describedabove, an image area corresponding to the parts box 30 a of the firstpart 20 a is set as a monitoring area P1 a immediately after thesetting. Further, the control unit 11 that performs the above monitoringarea setting corresponds to an example of the “monitoring area settingunit.”

In the monitoring area setting of the present embodiment, the worker Mmoves a parts box in a predetermined movement state while the imagingunit 13 is acquiring a video image. Then, an image difference occursaccording to the movement, and an area corresponding to the imagedifference in the sequential images is set and registered as amonitoring area. In the present embodiment, in order to narrow themovement range of the parts box for identifying the difference, as shownin FIG. 58, a movement state in which the parts box is repeatedly swungback and forth (hereinafter, also simply referred to as swinging state)as viewed from the worker M so as not to interrupt another parts box isadopted as the above predetermined movement state. In FIG. 58 and inFIG. 59 which will be described later, the parts boxes are illustratedin plan view as viewed from above, unlike the imaging state of FIG. 2.The parts accommodated in the respective parts boxes are omitted.

Referring to a flowchart of FIG. 57, the above monitoring area settingwill now be described in detail.

First, when the imaging unit 13 is ready to perform imaging (S1001 inFIG. 57), determination is performed at step S1003 to determine whetheror not setting start instruction has been issued. In the presentembodiment, for example, a predetermined gesture by the worker M is setas the above setting start instruction. Accordingly, when the worker Mperforms the predetermined gesture in the imaging range of the imagingunit 13, and the predetermined gesture is captured by the imaging unit13, it is determined that the setting start instruction has been issued(Yes at step S1003). For example, the above predetermined gesture may bea motion of displacing (swinging) at least some of the parts boxes intoa predetermined state, or may be a motion of causing the imaging unit 13to capture an image of an information code shown in the work instructionor the like or a marker of a predetermined shape.

Then, at step S1005, sequential image acquisition is performed toacquire sequential images corresponding to the video image of apredetermined period of time. Subsequently, at step S1007, imagecomparison is performed to compare images of each successive frame orseveral successive frames among the sequential images acquired asdescribed above to extract an area constituting the image difference(hereinafter, also simply referred to as difference area). Then, at stepS1009, determination is performed to determine whether or not adifference area larger than a predetermined range is generated. Until adifference area larger than a predetermined range is generated, it isdetermined as “No” and the steps from step S1005 onward are repeated.

Then, when it is determined as “Yes” at step S1009 due to a differencearea of a predetermined range or more being generated, hand areaacquisition is performed at step S1011. At step S1009, if the imagesinvolved in generation of the difference area include a region with ashape and size that can be generally recognized as a hand, this regionis obtained as a hand area.

Subsequently, at step S1013, determination is performed to determinewhether or not the parts box has been moved in a predetermined movementstate by the worker M. When the parts box held by the worker M is movedin the swinging state, the hand area acquired as described above andanother difference area, which corresponds to the parts box, areadjacent and continuous to each other. When the hand area and anotherdifference area are not adjacent to each other, the parts box isdetermined as not having been moved, and it is determined as “No” instep S1013. Then, the steps from step S1005 onward are repeated.Further, when the hand area is not acquired, it is also determined as“No” in step S1013. Then, the steps from step S1005 onward are repeated.

On the other hand, after the assembly order k is set to 1 at step S907in FIG. 56, when the parts box 30 a accommodating the first parts 20 aand held by the worker M is in the swinging state (see FIG. 58), adifference area is generated in the acquired sequential images at theportion of the hand holding the parts box 30 a and the portioncorresponding to the parts box 30 a. Accordingly, a hand area of thehand of the worker M holding the parts box 30 a is obtained, and, sincethe obtained hand area is adjacent to another difference area, whichcorresponds to a portion where the parts box 30 a is imaged, it isdetermined that the parts box has been moved in the predetermined stateby the worker M. Thus, it is determined as “Yes” at step S1013 FIG. 57.

In this case, storing of monitoring area is performed at step S1015, inwhich an area, which is obtained by adding the area presumed to be aportion held by the worker M to the difference area excluding the handarea, is set as the monitoring area P1 a of the parts box 30 a, andstored in the storage unit 12 (see FIG. 60A). At step S1015, themonitoring area can be accurately set by using the comparison result,among the above sequential images, between the image in which the partsbox is determined as being stopped as it is returned to the originalposition (image acquired in the state of FIG. 59A) and another imageacquired immediately before the above image (image acquired in the stateof FIG. 59B).

When the monitoring area P1 a is set for the parts box 30 aaccommodating the first parts 20 a, the monitoring area setting ends.Then, at step S907 in FIG. 56, determination is performed to determinewhether or not the assembly order k matches the total number N of theparts boxes, which is specified by the above work procedure information.In the above example, since the total number N is set to 4, it isdetermined as “No” at step S907. Then, after the assembly order k isincremented by 1 (S909), the steps from step S905 onward are repeated.Since the monitoring area has been set for the parts box 30 aaccommodating the first part 20 a, k is set to 2.

Then, monitoring area setting for the parts box 30 b accommodating thesecond parts 20 b starts. The monitoring area P1 b is set for the partsbox 30 b by swinging the parts box 30 b accommodating the second parts20 b in the same manner as that described above. Since k=N is notsatisfied (No at S907), k is set to 3 (S909). Then, the monitoring areaP1 c is set for the parts box 30 c by swinging the parts box 30 caccommodating the third parts 20 c (S905). Subsequently, since k=N isnot satisfied (No at S907), k is set to 4 (S909). Then, the monitoringarea P1 d is set for the parts box 30 d by swinging the parts box 30 daccommodating the fourth parts 20 d (S905).

Thus, when the monitoring areas P1 a to P1 d are set for four partsboxes 30 a to 30 d, respectively, (see FIG. 60B), k=N is satisfied.Then, it is determined as “Yes” at step S907 in FIG. 56, andnotification of setting completion is performed at step S911. At stepS911, the worker M is notified that the setting of all the monitoringareas has been completed for the manufacturing lot by modes of lightfrom the light emitting unit 15 or modes of sound from the speaker 16.Further, in the monitoring area setting, notification of individualsetting completion may also be performed by using the light emittingunit 15 or the speaker 16 each time when one monitoring area is set.

When completion of setting of the monitoring areas is notified asdescribed above, monitoring is performed at step S913. At step S913,dynamic detection is performed to the monitoring areas P1 a to P1 d setas described above to monitor the respective monitoring areas P1 a to P1d. Further, the control unit 11 that performs the dynamic detection cancorrespond to an example of a “detecting section.”

In the present embodiment, the dynamic detection is performed, as withthe monitoring area setting described above, by using a technique ofobserving a time-series change in the monitoring areas P1 a to P1 d byusing an image difference. Specifically, in the above monitoring, thedynamic detection of the monitoring area is performed by comparisonbetween the latest image acquired by the imaging unit 13 and anotherimage acquired earlier (for example, an image acquired immediatelybefore the latest image).

By the above dynamic detection, it is determined whether or not the handof the worker M has been detected by using a technique disclosed in, forexample, JP 2018-156279 A, on the basis of whether a change that hasoccurred in the monitoring area is of a size that can be generallyrecognized as a hand. Further, the control unit 11 that determineswhether or not the work according to a predetermined work procedure isperformed on the basis of the detection result of the above dynamicdetection can correspond to an example of a “determination unit.”

For example, when the worker M, who has finished swinging the partsboxes 30 a to 30 d, takes out the part 20 a from the parts box 30 a inorder to actually start the assembly work, the hand of the worker M isdetected in the monitoring area P1 a on the basis of the differencebetween the latest image and an image acquired immediately before thelatest image. When the detection result shows that the hand of theworker M is detected in the monitoring area P1 a, the monitoring area P1b, the monitoring area Plc, and the monitoring area P1 d in this orderin accordance with the above work procedure information, it isdetermined that the correct assembly work has been performed, and thusnormal work notification is performed as the determination result. Thisnormal notification is performed by, for example, lighting of greenlight, typically associated with correct answer, by the light emittingunit 15, or correct buzzer sound, typically associated with correctanswer, by the speaker 16. When receiving the normal notification, theworker M can recognize that the parts have been assembled in accordancewith the correct work procedure.

The normal notification may be performed for each monitoring area, ormay be performed when the hand of the worker M has been detected in allthe monitoring areas in accordance with the above work procedureinformation.

On the other hand, when the hand of the worker M is detected in themonitoring area in wrong order, for example, when the hand of the workerM is detected in the monitoring area P1 a, and the monitoring area P1 cin this order, it is determined that the wrong assembly work has beenperformed, and thus abnormal work notification is performed as thedetermination result. This abnormal notification is performed by, forexample, lighting of red light, typically associated with incorrectanswer, by the light emitting unit 15, or an alert message saying“please pick up the correct part” by the speaker 16. When receiving theabnormal notification, the worker M can immediately recognize that theparts have been taken out in an incorrect work procedure.

As described above, in the work support device 100 according to thepresent embodiment, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area (P1 a to P1 d) set by the monitoring area settingand a portion corresponding to the monitoring area in another image thatis acquired before the above image, taking out of the parts (20 a to 20d) from the parts boxes (30 a to 30 d) corresponding to the monitoringarea is detected. On the basis of the detection result, it is determinedwhether or not the work in accordance with the predetermined workprocedure is performed, and the determination result is notified to theworker M. In this process, in the monitoring area setting, on the basisof the images acquired by the imaging unit 13 when the parts boxes areindividually moved in a predetermined movement state by the worker M,the monitoring area corresponding to the parts box that is moved in thepredetermined movement state is set for each parts box.

A work support program according to the present embodiment is a programthat causes the control unit 11 of the work support device 100, whichsupports the work performed in accordance with a predetermined workprocedure by using the parts (20 a to 20 d) accommodated in a pluralityof parts boxes (30 a to 30 d), to execute: monitoring area setting(S905) for setting a monitoring area (P1 a to P1 d) for each parts boxin the imaging range of the imaging unit 13; detection (S913) fordetecting the parts being taken out from the parts boxes correspondingto the monitoring area, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area set by the monitoring area setting and a portioncorresponding to the monitoring area in another image that is acquiredbefore the above image; determination (S913) for determining whether ornot the work is performed in accordance with the predetermined workprocedure on the basis of the detection result by the above detection;and notification (S913) for notifying the determination result by theabove determination to the worker.

Further, in the monitoring area setting, on the basis of the imagesacquired by the imaging unit when the parts boxes are individually movedin a predetermined movement state by the worker, the monitoring areacorresponding to the parts box that is moved in the predeterminedmovement state is set for each parts box.

Thus, the monitoring area can be easily set for each parts box by theworker M simply moving the parts boxes individually in the predeterminedmovement state before starting the work. In particular, since the workerM does not need to perform mouse operation while watching the screen,erroneous setting of the monitoring area due to an operation error orthe like of the worker M can be avoided. Therefore, it is possible toset a monitoring area easily and correctly without imposing a workburden on the worker.

In addition, since the above predetermined movement state is a swingingstate that is repeated on the assumption that the parts box is returnedto the original position, the movement range of the parts box tends tobe small, and can be easily specified from the acquired images, themonitoring area can be more accurately set.

In particular, in the monitoring area setting, the monitoring area isset by using the comparison result, among the images acquired by theimaging unit 13, between the image in which the parts box is determinedas being stopped as it is returned to the original position (see FIG.59A) and another image acquired immediately before the above image (seeFIG. 59B). Thus, since the position of the parts box is located by usingthe difference between the image acquired immediately after the partsbox stops moving and the image acquired immediately before the parts boxstops moving, and the monitoring area is set according to the locatedposition of the parts box, the monitoring area can be set moreaccurately. In addition, since the monitoring area is set at the timingimmediately after the parts box stops moving, the time required forsetting can be reduced even in the setting of a number of monitoringareas, and thus the work efficiency can be improved.

Sixteenth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to a sixteenth embodiment will now be described.

The sixteenth embodiment differs from the fifteenth embodiment in thatsetting is performed according to the work of the worker withoutobtaining a predetermined work procedure (work procedure information) inadvance as a monitoring criterion. The components which aresubstantially the same as those of the fifteenth embodiment are denotedby the same reference signs, and the description thereof will beomitted.

In the fifteenth embodiment, since the work procedure information isread at the start of work support processing, it is necessary to performthe work of reading an information code provided in the work instructionor the like or the work of obtaining work procedure information from ahigher level device. Therefore, in the present embodiment, in order tofurther reduce the work load to the worker M in the work supportprocessing, a predetermined work procedure can be set according to theactual work by the worker M without obtaining the work procedureinformation in advance.

Hereinafter, in the present embodiment, referring to the flowchart inFIG. 61, the following detailed description will be given of the worksupport processing performed on the basis of the work support program,which is performed for each manufacturing lot by the control unit 11,when the worker M performs predetermined work by which parts containedin a plurality of parts boxes are individually assembled to theworkpiece W according to a predetermined work procedure.

The control unit 11 starts the work analysis processing when apredetermined start operation is performed to the operation unit 17.When the imaging unit 13 is ready to perform imaging (S1101 in FIG. 61),determination is performed at step S1103 to determine whether or notsetting start instruction has been issued. Then, when the abovepredetermined gesture is captured by the imaging unit 13, it isdetermined that the setting start instruction has been issued (Yes atS1103), and the assembly order k is set to 1 (S1105).

Then, at step S1107, sequential image acquisition is performed toacquire sequential images corresponding to the video image of apredetermined period of time. Subsequently, at step S1109, imagecomparison is performed to compare images of each successive frame orseveral successive frames among the sequential images acquired asdescribed above to extract a difference area. When the difference areagreater than the predetermined range is generated (Yes at S1111), handarea acquisition is performed at S1113. If the images involved ingeneration of the difference area include a region with a shape and sizethat can be generally recognized as a hand, this region is obtained as ahand area.

Then, in determination at step S1115, when the hand area obtained asdescribed above and another difference area corresponding to the partsbox are determined as not being adjacent to each other, it is determinedas “No,” and the steps from step S1107 are repeated. On the other hand,after the assembly order k is set to 1, when the parts box 30 aaccommodating the first parts 20 a and held by the worker M is in theswinging state, the hand area obtained as described above and anotherdifference area corresponding to a portion where the parts box 30 a isimaged are adjacent to each other. Accordingly, it is determined thatthe parts box has been moved in the predetermined state by the worker M,and it is determined as “Yes” at step S1115.

When determination at step S1115 is Yes, storing of monitoring area isperformed at step S1117, in which an area, which is obtained by addingthe area presumed to be a portion held by the worker M to the differencearea excluding the hand area, is set as the monitoring area P1 a of theparts box 30 a, and stored in the storage unit 12, as with the storingof monitoring area in fifteenth embodiment at step S1015 in FIG. 57.

Subsequently, at step S1119, determination is performed to determinewhether or not all the parts boxes used in the manufacturing lot havebeen moved in a predetermined movement state by the worker M. In thepresent embodiment, a predetermined end gesture is set to be performedto the work support device 100 after the worker M finished swinging theparts boxes individually. When the predetermined end gesture is notacquired by the imaging unit 13, it is determined as “No” at step S1119.Then, after the assembly order k is incremented by 1 (S1121), the stepsfrom step S1107 onward are repeated. In the present embodiment,repetition of the steps from step S1107 to step S1119 can correspond tothe monitoring area setting.

When the worker M, after having moved the last parts box, performs thepredetermined end gesture in the imaging range of the imaging unit 13,and the predetermined end gesture is captured by the imaging unit 13, itis determined that all the parts boxes have been moved in thepredetermined movement state (Yes at step S1119). When it is determinedas “Yes” at step S1119, notification of setting completion is performedat step S1123. As in notification of setting completion at step S911 inFIG. 56 according to the fifteenth embodiment, the worker M is notifiedthat the setting of all the monitoring areas has been completed for themanufacturing lot by modes of light from the light emitting unit 15 ormodes of sound from the speaker 16.

Further, the above predetermined end gesture may be, for example, amotion of causing the imaging unit 13 to capture an image of aninformation code shown in the work instruction or the like or a markerof a predetermined shape. Further, in determination at step S1119, forexample, an elapsed time after the worker M has individually swung theparts boxes can also be taken into consideration. In this case, when apredetermined time has elapsed after the last difference area wasdetected, it is determined that the worker M finished swinging the partsboxes individually, and thus it is determined as “Yes.”

When completion of setting of monitoring areas is notified as describedabove, work procedure setting is performed at step S1125. At step S1125,during actual assembly work performed by the worker M after the abovenotification, detection is performed to taking out of the parts inresponse to the above dynamic detection to thereby set the workprocedure on the basis of the order of taking out of the parts. When thework procedure is fixed as described above, the worker M, during theassembly work, takes out the parts from each parts box in the correctorder by repeating taking out from the parts box 30 a, the parts box 30b, the parts box 30 c, and the parts box 30 d in this order.Accordingly, by detecting this repetition in response to the abovedynamic detection, it is possible to recognize the correct workprocedure and set the work procedure as the work criterion. Further, thecontrol unit 11 that performs the above work procedure setting cancorrespond to an example of a “work procedure setting unit.”

When the work procedure is set as described above, monitoring isperformed at step S1127. At step S1127, as in the monitoring at stepS913 in FIG. 56, dynamic detection is performed to the monitoring areasP1 a to P1 d set as described above to monitor the respective monitoringareas P1 a to P1 d.

As described above, in the work support device 100 according to thepresent embodiment, after the plurality of monitoring areas (P1 a to P1d) are set by the monitoring area setting, a predetermined workprocedure is set by the work procedure setting (step S1125 in FIG. 61)performed by the control unit 11 on the basis of the order of taking outof the parts (20 a to 20 d) detected in response to the dynamicdetection.

Accordingly, after the monitoring area setting, since the predeterminedwork procedure can be automatically set by the worker M actually takingout the parts from the parts boxes to perform correct assembly, there isno need of performing the work of reading an information code providedin the work instruction or the like or the work of obtaining workprocedure information from a higher level device.

In setting of the monitoring areas, by swinging the parts boxes in thecorrect order of the work procedure, the correct work procedure can berecognized by the control unit 11. Accordingly, when the correct workprocedure is recognized in this manner, the work procedure setting atstep S1125 can be omitted.

The fifteenth and sixteenth embodiments, and modified examples accordingto the fifteenth and sixteenth embodiments are not limited to the onedescribed above, and may also be embodied as described below.

(1) The parts boxes 30, which are disposed within the imaging range ofthe imaging unit 13 as the monitoring target, are not limited to fourparts boxes 30 a to 30 d, and may also be one to three parts boxes, orfive or more parts boxes. Further, the parts boxes 30 are not limited tobeing horizontally arranged side by side within the imaging range of theimaging unit 13, and, for example, may also be vertically arranged atmultiple levels such as upper and lower levels. Moreover, the parts box30 to be monitored is not limited to being formed as a shape with arectangular opening edge. For example, the opening edge may be formed ina trapezoidal shape, or the opening edge may be partially curved.Further, the parts box 30 to be monitored may also be configured, forexample, as a storage bag such that the shape of the opening edge variesdepending on how it is placed.

(2) In the monitoring area setting according to the fifteenth andsixteenth embodiments, the predetermined movement state is not limitedto the swinging state. Any movement state, by which the image differencecan be easily obtained, such as lifting the parts box, may also beadopted.

(3) In the fifteenth and sixteenth embodiments, the invention can beapplied not only to work support in assembly of the parts to theworkpiece W such as a printed circuit board according to a predeterminedwork procedure, but also to work support in assembly of the parts to asemi-finished product, for example, in an assembly line for high-mixlow-volume products according to a predetermined work procedure.

Seventeenth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to a seventeenth embodiment will now be described.

The seventeenth embodiment, as in the fifteenth and sixteenthembodiments, mainly differs from the first embodiment in the process forsetting the monitoring area. The components which are substantially thesame as those of the above embodiments are denoted by the same referencesigns, and the description thereof will be omitted.

As illustrated in FIG. 54, the work support device 100 according to thepresent embodiment, as in the fifteenth embodiment, is installed on thework table 1 or the like, and is configured as a device that supportsassembly work performed by the worker M by using parts contained in aplurality of parts boxes according to a predetermined work procedure.

FIG. 62 is a more detailed view of the parts boxes 30 a shown in FIG. 2.The parts box 30 a is formed such that an outer periphery of arectangular bottom wall 31 is connected to a lower end of a peripheralwall 32, and the peripheral wall 32 has a rectangular ring shape, whoseupper end faces 33 a to 33 d are provided with four corners 34 a to 34d. The parts boxes 30 b to 30 d, as with the parts box 30 a, are eachformed such that an outer periphery of a rectangular bottom wall isconnected to a lower end of a peripheral wall, and the peripheral wallhas a rectangular ring shape, whose upper end faces are provided withfour corners.

Referring to a flowchart of FIGS. 63 and 64, the work support processingin seventeenth embodiment will now be described in detail. In theexample of the predetermined work procedure described below, the workprocedure is fixed in advance, by which the part 20 a in the parts box30 a, the part 20 b in the parts box 30 b, the part 20 c in the partsbox 30 c, and the part 20 d in the parts box 30 d are sequentiallyassembled to the workpiece W in this order. The processing shown in FIG.63 is the same as that in FIG. 58 except for the monitoring area settingat step S905A. Therefore, the description of the same steps in FIG. 63as those in FIG. 58 will be omitted.

In the monitoring area setting of the present embodiment, the worker Mtouches one of the corners on the upper end face of the peripheral wallof the parts box with a finger F while the imaging unit 13 is acquiringan image. Then, starting from the corner, a boundary along the upper endface is detected so that an area having a rectangular shape surroundedby the boundary is set and registered as a monitoring area.

Referring to a flowchart of FIG. 64, the above monitoring area settingfor setting a monitoring area along the boundary, starting from thecorner of the parts box where the finger has touched, will now bedescribed in detail.

First, when the imaging unit 13 is ready to perform imaging (S1201 inFIG. 64), determination is performed at step S1203 to determine whetheror not a human finger is detected in the captured image. It isdetermined as “No” repeatedly until a human finger is detected.

When a human finger is detected in the captured image during thisrepetition (Yes at S1203), it is determined whether the detected fingeris stationary or not in determination at step S1205. When the finger ismoving (No at S1205), the steps from step S1203 onward are performed.While movement of the finger is detected, determination of Yes at stepS1203 and determination of No at step S1205 are repeated.

When the detected finger becomes stationary, it is determined as “Yes”in determination at step S1205, and measurement of stationary timeduring which the detected finger is stationary starts (S1207).Subsequently, in determination at step S1209, it is determined whetheror not the stationary time has passed a predetermined time set inadvance (for example, 5s). When the stationary time has not passed thepredetermined time (No at S1209), it is determined whether or not thedetected finger stays stationary in determination at step S1211. Whenthe detected finger stays stationary (Yes at S1211), the steps from stepS1209 are performed. Then, when movement of the finger is detectedbefore the stationary time has passed the above predetermined time (Noat S1211), the steps from step S1203 are performed.

On the other hand, as shown in FIG. 65, when the finger F of the workerM is stationary while being in contact with the corner 34 a, which islocated between the upper end face 33 a and the upper end face 33 d ofthe parts box 30 a, and the stationary time of the finger F has passedthe above predetermined time (Yes at S1209), start point detection isperformed at step S1213. At step S1213, in the latest captured image setas an image of a search target, a point regarded as an intersectionbetween two straight lines extending from a position of the finger Fwhich is stationary (two straight lines which will intersect at aposition closest to the coordinates of the finger tip) is detected as astart point. As described above, when the finger F is stationary whilebeing in contact with the corner 34 a, a line segment corresponding tothe upper end face 33 a and a line segment corresponding to the upperend face 33 d extend from a position of the finger F which isstationary. Accordingly, the corner 34 a where the finger F is incontact is detected as a start point.

Then, at step S1215, search target setting is performed to set one ofthe two straight lines extending from the finger F which is stationaryas a search target. Subsequently, in search at step S1217, anintersection which is a tip of the straight line that is set as thesearch target and intersects another straight line is searched in thecaptured image.

In the present embodiment, a direction of the turn at the intersectionin plan view of the parts box as viewed from above in search isdetermined to be clockwise, so one of the two straight lines detected atthe start of search, which is directed in the clockwise direction, isset as a first search target. As described above, when the finger F isstationary at the corner 34 a of the parts box 30 a, part of a linesegment corresponding to the upper end face 33 a, and part of a linesegment corresponding to the upper end face 33 d are detected as twostraight lines extending from the finger F. Accordingly, a line segmentcorresponding to the upper end face 33 a, which is directed in theclockwise direction, is set as a search target (see the arrow Sa in FIG.65).

In search at step S1217 in FIG. 64, when a tip of the straight line ofthe search target (intersection) is detected, determination is performedat step S1219 to determine whether or not the detected intersectionmatches the above start point. Here, when an intersection that is a tipof the line segment corresponding to the upper end face 33 a and thatintersects the line segment corresponding to the upper end face 33 b issearched, this intersection corresponds to the corner 34 b and does notmatch the above start point. Accordingly, it is determined as “No” atstep S1219.

When determination is No at step S1219, search target switching isperformed at step S1221 so that another straight line extending from thesearched intersection is set as a search target. As described above,when the intersection corresponding to the corner 34 b is searched,switching is performed so that the line segment corresponding to theupper end face 33 b is set as a search target (see the arrow Sb in FIG.65). Then at step S1217 in FIG. 64, when an intersection that is a tipof the line segment corresponding to the upper end face 33 b and thatintersects the line segment corresponding to the upper end face 33 c issearched, this intersection corresponds to the corner 34 c and does notmatch the above start point. Accordingly, it is determined as “No” atstep S1219. Then, switching is performed so that the line segmentcorresponding to the upper end face 33 c is set as a search target(S1221: see the arrow Sc in FIG. 65), and the intersection that is a tipof the line segment corresponding to the upper end face 33 c and thatintersects the line segment corresponding to the upper end face 33 d issearched (S1217). Since the intersection corresponds to the corner 34 dand does not match the above start point (No at S1219), switching isperformed so that the line segment corresponding to the upper end face33 d is set as a search target (S1221: see the arrow Sd in FIG. 65), andthe intersection that is a tip of the line segment corresponding to theupper end face 33 d and that intersects the line segment correspondingto the upper end face 33 a is searched (S1217). Since this intersectioncorresponds to the corner 34 a and matches the above start point, it isdetermined as “Yes” in determination at step S1219.

When it is determined as “Yes” at step S1219, boundary setting isperformed at step S1223. At step S1223, a polygonal ring formed byconnecting the line segments that have been searched is detected and setas a boundary, which extends from the start point, turns at a pluralityof intersections (i.e., the corners), and then returns to the startpoint. The polygonal area surrounded by the boundary is set as amonitoring area. As described above, when the boundary formed byconnecting the line segments corresponding to the upper end faces 33 ato 33 d is detected and set, the rectangular area surrounded by theboundary is set as the monitoring area P1 a for the parts box 30 aaccommodating the first parts 20 a (see FIG. 60A). Therefore, in FIG.60A, the boundary is denoted by reference numeral P1 a. Similarly, whenthe finger F of the worker M is stationary while being in contact withthe corner 34 b (34 c, 34 d) of the parts box 30 a, the rectangular areasurrounded by the boundary, which extends from the corner 34 b (34 c, 34d) taken as a start point, turns at a plurality of intersections, andthen returns to the start point, is also set as the monitoring area P1a.

Here, with reference to FIG. 66, the reason that the direction of a turnat the intersection in search is set to a first direction (clockwisedirection in plan view of the parts box as viewed from above) will nowbe specifically described.

As shown in FIG. 66, for example, in setting of the monitoring area ofthe parts box 30 e, it is assumed that the parts box 30 e and the partsbox 30 f are arranged side by side with the peripheral wall of the partsbox 30 e and the peripheral wall of the parts box 30 f in contact witheach other. In the above search, if the upper end face of the parts box30 e is determined as being continuous to the upper end face of theparts box 30 f at the position adjacent to the parts box 30 f, anunintended intersection (see reference numeral Sp1 in FIG. 66) may bedetected. In this case, the upper end face of the parts box 30 f may beerroneously searched as a straight line extending from the intersection(see the arrow Sf in FIG. 66). Meanwhile, for example, while theboundary is defined to turn in the clockwise direction at theintersection in plan view of the parts box as viewed from above, thedirection of the turn at the intersection Sp1 along the upper end faceof the parts box 30 f is a counterclockwise direction, which is oppositeto the clockwise direction.

Therefore, according to the present embodiment, the direction of a turnat the intersection in search is set to clockwise direction as describedabove (one direction). As shown in FIG. 66, a straight line extendingfrom the unintended intersection Sp1 and corresponding to the upper endface of the parts box 30 e (see the arrow Se in FIG. 66) is directed inthe clockwise direction from the intersection Sp1, whereas a straightline extending from the unintended intersection Sp1 and corresponding tothe upper end face of the parts box 30 f (see the arrow Sf in FIG. 66)is directed in the counterclockwise direction from the intersection Sp1.Accordingly, the upper end face of the parts box 30 f, which extends inthe counterclockwise direction, is prevented from being erroneously setas a search target.

When the monitoring area P1 a is set for the parts box 30 aaccommodating the first parts 20 a, the monitoring area setting ends.Then, at step S907 in FIG. 63, determination is performed to determinewhether or not the assembly order k matches the total number N of theparts boxes, which is specified by the above work procedure information.

In the above example, since the total number N is set to 4, it isdetermined as “No” at step S907. Then, after the assembly order k isincremented by 1 (S909), the steps from step S905A onward are repeated.Since the monitoring area has been set for the parts box 30 aaccommodating the first part 20 a, k is set to 2. Then, monitoring areasetting for the parts box 30 b accommodating the second parts 20 bstarts. The monitoring area P1 b is set for the parts box 30 b when thefinger is stationary while being in contact with one of the corners ofthe parts box 30 b accommodating the second parts 20 b in the samemanner as that described above. Since k=N is not satisfied (No at S907),k is set to 3 (S909).

Then, the monitoring area P1 c is set for the parts box 30 c when thefinger is stationary while being in contact with one of the corners ofthe parts box 30 c accommodating the third parts 20 c (5905A). Since k=Nis not satisfied (No at S907), k is set to 4 (S909). Then, themonitoring area P1 d is set for the parts box 30 d when the finger isstationary while being in contact with one of the corners of the partsbox 30 d accommodating the fourth parts 20 d (5905A).

As described above, in the work support device 100 according to thepresent embodiment, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area (P1 a to P1 d) set by the monitoring area settingand a portion corresponding to the monitoring area in another image thatis acquired before the above image, taking out of the parts (20 a to 20d) from the parts boxes (30 a to 30 d) corresponding to the monitoringarea is detected. On the basis of the detection result, it is determinedwhether or not the work in accordance with the predetermined workprocedure is performed, and the determination result is notified to theworker M. In this process, in the monitoring area setting, a boundary(P1 a to P1 d) is detected in the image acquired by the imaging unit 13.The boundary (P1 a to P1 d) extends from a start point at the cornerwhere the finger F of the worker M is in contact, among the plurality ofcorners (34 a to 34 d) on the upper end faces (33 a to 33 d) of theperipheral wall of the parts boxes, turns at a plurality ofintersections, and then returns to the start point. The polygonal areasurrounded by the boundary is set and registered as a monitoring areafor each parts box.

A work support program according to the present embodiment is a programthat causes the control unit 11 of the work support device 100, whichsupports the work performed in accordance with a predetermined workprocedure by using the parts (20 a to 20 d) accommodated in a pluralityof parts boxes (30 a to 30 d), to execute: monitoring area setting(5905A) for setting a monitoring area (P1 a to P1 d) for each parts boxin the imaging range of the imaging unit 13; detection (S913) fordetecting the parts being taken out from the parts boxes correspondingto the monitoring area, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area set by the monitoring area setting and a portioncorresponding to the monitoring area in another image that is acquiredbefore the above image; determination (S913) for determining whether ornot the work is performed in accordance with the predetermined workprocedure on the basis of the detection result by the above detection;and notification (S913) for notifying the determination result by theabove determination to the worker. In the monitoring area setting, aboundary (P1 a to P1 d) is detected in the image acquired by the imagingunit 13. The boundary (P1 a to P1 d) extends from a start point at thecorner where the finger F of the worker M is in contact, among theplurality of corners 34 a to 34 d on the upper end faces 33 a to 33 d ofthe peripheral wall 32 of the parts boxes, turns at a plurality ofintersections, and then returns to the start point. The polygonal areasurrounded by the boundary is set and registered as a monitoring areafor each parts box.

Thus, the monitoring area can be easily set for each parts box by theworker M, when touching, with the finger F, one of the corners on theupper end face of the peripheral wall of the parts box before startingthe work. In particular, since the worker M does not need to performmouse operation while watching the screen, erroneous setting of themonitoring area due to an operation error or the like of the worker Mcan be avoided. Therefore, it is possible to set a monitoring areaeasily and correctly without imposing a work burden on the worker.

In particular, in the monitoring area setting, since the boundary (P1 ato P1 d) to be detected is set to turn at the intersection in onedirection, the line extending from the unintended intersection (Sp1) inthe direction different from the first direction can be excluded as aline extending along the upper end face of the peripheral wall of theadjacent parts box.

Eighteenth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to an eighteenth embodiment will now be described.

The eighteenth embodiment mainly differs from the seventeenth embodimentin that, in monitoring area setting, a finger in a stationary statetouches a plurality of corners for each parts box. The components whichare substantially the same as those of the seventeenth embodiment aredenoted by the same reference signs, and the description thereof will beomitted.

In the present embodiment, a finger in a stationary state touches aplurality of corners for each parts box to thereby improve detectionaccuracy of the boundary for setting the monitoring area. Specifically,in the monitoring area setting, a boundary is detected by setting one ofa plurality of corners, which is touched by the finger F in a stationarystate as a start point, and the other corner as at least part of theintersections. Accordingly, the boundary that returns to the originalstart point without passing through the other corner, as anintersection, touched by the finger F in a stationary state can bedetermined as an erroneous boundary. Thus, detection accuracy of theboundary can be enhanced. For example, as shown in FIG. 67, when thefinger F in a stationary state touches the corner 34 a and the corner 34c of the parts box 30 a in this order, the boundary having a start pointat the corner 34 a (corner 34 c) can be determined as an erroneouslydetected boundary if it does not have a position corresponding to thecorner 34 c (corner 34 a) as an intersection.

Further, detection accuracy of the boundary for setting the monitoringarea can be enhanced by determining in advance the order that the fingerF in a stationary state should touch the corner corresponding to theintersection to be detected. For example, it is determined in advancethat the finger F in a stationary state first touches a cornercorresponding to the intersection to be detected second, and thentouches a corner which is set as a start point. Then, as shown in FIG.67, the finger F in a stationary state first touches the corner 34 c ofthe parts box 30 a, and then touches the corner 34 a. Thus, even if anunintended intersection Sp2 is detected due to the part 20 a in theparts box 30 a, the intersection Sp2 is not erroneously searched as aposition where to turn since the second intersection is set to aposition corresponding to the corner 34 c. Therefore, detection accuracyof the boundary for setting the monitoring area can be enhanced. In FIG.67, the parts box 30 a is shown in plan view, and one part 20 a isexaggerated for convenience.

Nineteenth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to a nineteenth embodiment will now be described.

The nineteenth embodiment mainly differs from the seventeenth embodimentin that, in monitoring area setting, a boundary is detected as a ringshape having a predetermined width. The components which aresubstantially the same as those of the seventeenth embodiment aredenoted by the same reference signs, and the description thereof will beomitted.

In the present embodiment, an upper end face of the peripheral wall ofthe parts box is detected as a ring shape having a predetermined width,and the inner edge of the ring is searched. Specifically, when thecorner 34 a of the parts box 30 a is touched by the finger F in thestationary state, in start point detection at step S1213 in FIG. 64, asshown in FIG. 68, a point regarded as an intersection SP1 a between aline segment corresponding to an inner edge 35 a of an upper end face 33a and a line segment corresponding to an inner edge 35 d of an upper endface 33 d. Then, in search target setting at step S1215, first, a linesegment corresponding to the inner edge 35 a of the upper end face 33 aand the intersection SP1 b at the tip of the line segment are set as asearch target. Then, in search target switching at step S1221, thesearch target is switched from a line segment corresponding to the inneredge 35 b of the upper end face 33 b and the intersection SP1 c at thetip of the line segment, to a line segment corresponding to the inneredge 35 c of the upper end face 33 c and the intersection SP1 d at thetip of the line segment, and a line segment corresponding to the inneredge 35 d of the upper end face 33 d, in this order. In FIG. 68, theparts boxes 30 a and 30 b are shown in plan view, in which the parts 20a and 20 b are omitted and the width of the upper end faces of the partsboxes 30 a and 30 b are exaggerated for convenience.

Thus, in boundary setting at step S1223 in FIG. 64, the rectangular areasurrounded by the inner edges 35 a to 35 d is set as the monitoring areaP1 a for the parts box 30 a. Therefore, since the boundary is detectedas a ring shape having a predetermined width, and a polygonal areasurrounded by the inner edge (35 a to 35 d) of the ring is set as amonitoring area, the parts box (30 b), which may be adjacent andcontinuous to the outer edge of the ring shape as shown in FIG. 68 isprevented from affecting the setting of the monitoring area (P1 a), andthus the monitoring area can be more accurately set.

Twentieth Embodiment

With reference to the drawings, a work support device and a work supportprogram according to a twentieth embodiment will now be described.

The twentieth embodiment mainly differs from the seventeenth embodimentin that, in monitoring area setting, a monitoring area is set accordingto a ring-shaped trajectory which is drawn by the worker's fingertracing the upper end face of the peripheral wall of the parts box. Thecomponents which are substantially the same as those of the seventeenthembodiment are denoted by the same reference signs, and the descriptionthereof will be omitted.

In the monitoring area setting of the present embodiment, when an areasurrounded by the ring-shaped trajectory which is drawn by the finger Fof the worker M tracing the upper end face of the peripheral wall of theparts box is imaged by the imaging unit 13, the area surrounded by thering-shaped trajectory is set as a monitoring area. That is, eachmonitoring area can be set by tracing the upper end face of theperipheral wall of each parts box.

Referring to a flowchart of FIG. 69, the monitoring area setting in thepresent embodiment will now be described in detail.

As in the above seventeenth embodiment, when the stationary time of thefinger F of the worker M in contact with the corner of the parts box haspassed the above predetermined time (Yes at S1209 in FIG. 69), and thecorner that is in contact with the finger F is detected as a start point(S1213), trajectory imaging is performed at step S1225. At step S1225,as the finger in a stationary state starts moving to draw a ring shape,imaging of the trajectory of the movement is performed.

As shown in FIG. 70, for example, when the worker M traces, with thefinger F, from a start point at the corner 34 a of the parts box 30 a,and then along the upper end face 33 a, the corner 34 b, the upper endface 33 b, the corner 34 c, the upper end face 33 c, the corner 34 d,the upper end face 33 d, and the corner 34 a in this order, aring-shaped trajectory (P1 a) is imaged. The area surrounded by the ringshape trajectory is set as the monitoring area P1 a. Similarly, when theworker M traces the upper end face of the peripheral wall of the partsbox 30 b with the finger F, the monitoring area P1 b is set. When theworker M traces the upper end face of the peripheral wall of the partsbox 30 c, the monitoring area P1 c is set. When the worker M traces theupper end face of the peripheral wall of the parts box 30 d, themonitoring area P1 d is set. In FIG. 70, the parts box 30 a is shown inplan view, and the part 20 a is omitted for convenience.

As described above, in the work support device 100 according to thepresent embodiment, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area (P1 a to P1 d) set by the monitoring area settingand a portion corresponding to the monitoring area in another image thatis acquired before the above image, taking out of the parts (20 a to 20d) from the parts boxes (30 a to 30 d) corresponding to the monitoringarea is detected. On the basis of the detection result, it is determinedwhether or not the work in accordance with the predetermined workprocedure is performed, and the determination result is notified to theworker M. In this process, in the monitoring area setting, when an areasurrounded by the ring-shaped trajectory which is drawn by the finger Fof the worker M tracing the upper end face (33 a to 33 d) of theperipheral wall of the parts box is imaged by the imaging unit 13, thearea surrounded by the ring-shaped trajectory is set as a monitoringarea for each parts box.

A work support program according to the present embodiment is a programthat causes the control unit 11 of the work support device 100, whichsupports the work performed in accordance with a predetermined workprocedure by using the parts (20 a to 20 d) accommodated in a pluralityof parts boxes (30 a to 30 d), to execute: monitoring area setting(S905A) for setting a monitoring area (P1 a to P1 d) for each parts boxin the imaging range of the imaging unit 13; detection (S913) fordetecting the parts being taken out from the parts boxes correspondingto the monitoring area, on the basis of the comparison result, among theimages captured by the imaging unit 13, between a portion correspondingto the monitoring area set by the monitoring area setting and a portioncorresponding to the monitoring area in another image that is acquiredbefore the above image; determination (S913) for determining whether ornot the work is performed in accordance with the predetermined workprocedure on the basis of the detection result by the above detection;and notification (S913) for notifying the determination result by theabove determination to the worker.

In the monitoring area setting, when an area surrounded by thering-shaped trajectory which is drawn by the finger F of the worker Mtracing the upper end face (33 a to 33 d) of the peripheral wall of theparts box is imaged by the imaging unit 13, the area surrounded by thering-shaped trajectory is set as a monitoring area for each parts box.

Thus, the monitoring area can be easily set for each parts box by theworker M, when tracing, with the finger F, the upper end face of theperipheral wall of the parts box before starting the work. Inparticular, since the worker does not need to perform mouse operationwhile watching the screen, erroneous setting of the monitoring area dueto an operation error or the like of the worker can be avoided.Therefore, it is possible to set a monitoring area easily and correctlywithout imposing a work burden on the worker.

Further, in the monitoring area setting of the present embodiment, aconfiguration is also possible in which, the area surrounded by thering-shaped trajectory is set as a monitoring area when the ring-shapedtrajectory drawn by the finger F of the worker M is imaged withoutdetecting the finger F of the worker M being stationary.

The present invention is not limited to the seventeenth to twentiethembodiments and the modified examples of the seventeenth to twentiethembodiments, and, for example, may be embodied as described below.

(1) The parts boxes 30, which are disposed within the imaging range ofthe imaging unit 13 as the monitoring target, are not limited to fourparts boxes 30 a to 30 d, and may also be one to three parts boxes, orfive or more parts boxes. Further, the parts boxes 30 are not limited tobeing horizontally arranged side by side within the imaging range of theimaging unit 13, and, for example, may also be vertically arranged atmultiple levels such as upper and lower levels. Further, the upper endface of the peripheral wall of the parts box 30 to be monitored is notlimited to being formed in a rectangular ring shape having four corners34 a to 34 d, and, for example, may be formed in a polygonal ring shapehaving a plurality of corners, such as a trapezoidal shape. Further, theparts box 30 to be monitored may also be configured, for example, as astorage bag such that the shape of the upper end face of the peripheralwall varies depending on how it is placed as long as the upper end faceof the peripheral wall has a shape that is recognizable from thecaptured image as a ring-shaped line or a ring shape having apredetermined width.

(2) The invention according to the seventeenth to twentieth embodimentscan be applied not only to work support in assembly of the parts to theworkpiece W such as a printed circuit board according to a predeterminedwork procedure, but also to work support in assembly of the parts to asemi-finished product, for example, in an assembly line for high-mixlow-volume products according to a predetermined work procedure.

PARTIAL REFERENCE SIGNS LIST

-   10 . . . work analysis device-   11 . . . control unit (setting unit, generation unit, monitoring    area setting unit, reliability setting unit, determination unit,    delimitation information setting unit, detection section, work    procedure setting unit)-   13 . . . imaging unit-   20, 20 a to 20 d . . . parts-   30, 30 a to 30 d . . . parts box-   31 a to 31 d . . . markers-   32 . . . peripheral wall-   33 a to 33 d . . . upper end face-   34 a to 34 d . . . corner-   35 a to 35 d . . . inner edge-   100 . . . work support device-   F . . . finger-   P1 a to Ptd, P2 a to Ptd, P11 a to P11 d, P21 a to P21 d . . .    monitoring area-   W . . . work (assembly target)-   Wa˜Wd . . . assembly position

1-24. (canceled)
 25. A work analysis device which generatesdetermination data for determining whether or not a worker's manual workis now repeatedly performed according to a predetermined work procedure,the work analysis device comprising: an imaging unit imaging, as a workvideo, a state where a plurality of fundamental work operations areperformed repeatedly and manually in a predetermined order by theworker; a setting unit setting delimitation information for delimitingthe work video at detection timings at each of which a predeterminedmotion is detected, based on the predetermined motion which ispreviously set for each of the fundamental work operations; and ageneration unit generating the determination data which includes boththe work video and the delimitation information.
 26. The work analysisdevice of claim 25, wherein the fundamental work operations compose workin which parts taken out of parts boxes assigned to the fundamental workoperations are assembled with an assembly target; and the predeterminedmotion is assigned to a motion for taking the parts from the partsboxes.
 27. The work analysis device of claim 25, wherein the fundamentalwork operations compose work in which parts taken out of parts boxesassigned to the fundamental work operations are assembled with anassembly target; and the predetermined motion is assigned to a motionfor transferring the parts to an assembly position of the assemblytarget.
 28. The work analysis device of claim 25, wherein the generationunit is configured to calculate a normal range of work time for each ofthe fundamental work operations based on the delimitation information,and to generate, as the determination data, the fundamental workoperation whose work time is within the normal range.
 29. The workanalysis device of claim 28, wherein the generation unit is configuredto generate the determination data such that one or more of thefundamental work operations, of which work time is out of the normalrange, are excluded from the determination data.
 30. The work analysisdevice of claim 25, wherein the generation unit is configured togenerate the determination data such that, of the fundamental workoperations, a fundamental work operation performed immediately before afundamental work operation performed in an order different from thepredetermined order is excluded from the determination data, based onthe delimitation information.
 31. A work analysis device which generatesa work video for a work analysis by repeatedly imaging a predeterminedwork in which a plurality of fundamental work operations are repeatedlyperformed with worker's manual work in a predetermined order, the devicecomprising: an imaging unit imaging the predetermined work repeatedlyperformed by the worker; a monitoring area setting unit setting aplurality of monitoring areas including a first monitoring area fordetecting a first operation among the fundamental work operations and asecond operation among the fundamental work operations, for each of thefundamental work operations in an imaging range of the imaging unit; areliability setting unit setting, for each of the monitoring areas,reliability such that the reliability increases with an increase in apossibility that a motion related to the fundamental work operations isperformed in the monitoring area, based on a comparison made between aportion corresponding to the monitoring area in an image captured by theimaging unit and a portion corresponding to the monitoring area in afurther image captured by the imaging unit, the further image beingcaptured prior to the image; a determination unit determining whether ornot a mutually corresponding fundamental work operation among thefundamental work operations is performed, based on the reliabilitieswhich are set to the first and second monitoring areas; and adelimitation information setting unit setting delimitation informationwhich enables a work video imaged by the imaging unit to be delimitedevery fundamental work operation determined to be perfumed by thedetermination unit, based on at least one of a timing at which the imagefor setting the reliability is set for the first monitoring area and atiming at which the image for setting the reliability is set for thesecond monitoring area.
 32. The work analysis device of claim 31,wherein the fundamental work operations are assigned to work for takingparts out of predetermined parts boxes and assembling the taken partswith an assembly target, the first operation is assigned to an operationfor taking the parts from the parts boxes, and the second operation isassigned to an operation for assembling the taken parts at an assemblyposition of the assembly target.
 33. The work analysis device of claim32, wherein the reliability setting unit is configured to change how toset the reliability for the first monitoring area depending on whetheror not the parts are countable based on a difference between imagescaptured by the imaging unit in the first monitoring area.
 34. The workanalysis device of claim 32, wherein the determination unit isconfigured to determine whether or not the mutually correspondingfundamental work operation is performed, based on a sum of quantifiedvalues of all the reliabilities produced such that the reliability forthe second monitoring area is weighted more than the reliability for thefirst monitoring area.
 35. A work support device which supports workperformed in a predetermined work procedure, the work involving takingout parts accommodated in a plurality of parts boxes, the devicecomprising: an imaging unit; a monitoring area setting unit settingmonitoring areas respectively to the parts boxes within an imaging rangeof the imaging unit; a detection section detecting a worker's takingaction of the parts from each of the parts boxes, based on a comparisonmade between a portion corresponding to the monitoring area in adesignated image captured by the imaging unit and a portioncorresponding to the monitoring area in a further image captured by theimaging unit, the further image being captured prior to the designatedimage; a determination unit determining whether or not the workaccording to the predetermined work procedure is performed, based on adetection result provided by the detection section; and a notificationunit notifying the worker of a determined result provided by thedetermination unit, wherein the monitoring area setting unit isconfigured to set, based on an image captured by the imaging unit, themonitoring areas every one of the parts boxes, the monitoring aerascorresponding to the parts boxes which are moved in a predeterminedmovement state, when the parts boxes are individually moved in thepredetermined movement state.
 36. The work support device of claim 35,wherein the predetermined movement state is a swinging state of one ormore of the parts boxes that are repeated on the assumption that theparts boxes are returned to original positions thereof.
 37. The worksupport device of claim 36, wherein the monitoring area setting unit isconfigured to set the monitoring areas, based on a comparison between adesignated image showing that the parts boxes are determined to bestopped at the original positions due to the return thereof and afurther image captured immediately before capturing the designatedimage, the designated and further images belonging to images captured bythe imaging unit.
 38. The work support device of claim 35, comprising awork procedure setting unit configured to set the predetermined workprocedure based on an order of taking out of the parts which is detectedby the detection section, after the plurality of monitoring areas areset by the monitoring area setting unit.
 39. A work support device whichsupports work performed in a predetermined work procedure, the workinvolving taking out parts accommodated in a plurality of parts boxes,the device comprising: an imaging unit; a monitoring area setting unitsetting monitoring areas respectively to the parts boxes within animaging range of the imaging unit; a detection section detecting aworker's taking action of the parts from each of the parts boxes, basedon a comparison made between a portion corresponding to the monitoringarea in a designated image captured by the imaging unit and a portioncorresponding to the monitoring area in a further image captured by theimaging unit, the further image being captured prior to the designatedimage; a determination unit determining whether or not the workaccording to the predetermined work procedure is performed, based on adetection result provided by the detection section; and a notificationunit notifying the worker of a determined result provided by thedetermination unit, wherein each of the parts boxes has a peripheralwall having an upper end face, the upper end face being formed into apolygonal ring shape with a plurality of corners, and the monitoringarea setting unit is configured to i) detect a boundary in an imagecaptured by the imaging unit, the boundary starting from a stat pointdesigned by a worker's finger touched to one of the corners, extendingalong line segments of the polygonal ring shape with a turn at each of aplurality of intersections configured by two of the corners, and returnto the start point, and ii) set a polygonal ring shaped area surroundedby the boundary as the monitoring area for each of the parts boxes. 40.The work support device of claim 39, wherein the monitoring area settingunit is configured to detect the boundary such that at least part of thecorners forms the start point and the intersections.
 41. The worksupport device of claim 39, wherein the monitoring area setting unit isconfigured to detect the boundary such that the boundary turns at theintersections in one way.
 42. The work support device of claim 39,wherein the monitoring area setting unit is configured to detect, as theboundary, a ring shape portion having a predetermined width, and set, asthe monitoring area, a polygonal area surrounded by an inner edge of thering shape portion.
 43. A work support device which supports workperformed in a predetermined work procedure, the work involving takingout parts accommodated in a plurality of parts boxes, the devicecomprising: an imaging unit; a monitoring area setting unit settingmonitoring areas respectively to the parts boxes within an imaging rangeof the imaging unit; a detection section detecting a worker's takingaction of the parts from each of the parts boxes, based on a comparisonmade between a portion corresponding to the monitoring area in adesignated image captured by the imaging unit and a portioncorresponding to the monitoring area in a further image captured by theimaging unit, the further image being captured prior to the designatedimage; a determination unit determining whether or not the workaccording to the predetermined work procedure is performed, based on adetection result provided by the detection section; and a notificationunit notifying the worker of a determined result provided by thedetermination unit, wherein the monitoring area setting unit isconfigured to i) respond to a state where the imaging unit captures aring-shaped trajectory drawn by a worker's finger which traces an upperend face of a peripheral wall of each of the parts boxes, and ii) set,every one of the parts boxes, the monitoring area which is an areasurround by the ring-shaped trajectory.
 44. A work analysis method whichgenerates determination data for determining whether or not a worker'smanual work is now repeatedly performed according to a predeterminedwork procedure, the work analysis method comprising: making an imagingunit image, as a work video, a state where a plurality of fundamentalwork operations are performed repeatedly and manually in a predeterminedorder by a worker; setting delimitation information for delimiting thework video at detection timings at each of which a predetermined motionis detected, based on the predetermined motion which is previously setfor each of the fundamental work operations; and generating thedetermination data which includes both the work video and thedelimitation information.